How to build an RSS feed-based News app with Streamlit, Heroku and Android Studio?
Build an Android app that aggregates news from your favorite news websites. This article is for Python developers and needs no Android development experience.
What are RSS Feeds?
RSS stands for Really Simply Syndication. RSS feeds are standardized XML files that enable users to access updates from websites. Below is the structure of a typical RSS feed. RSS feeds may also have a few additional elements like image of the news article, etc. In this article, we’ll use RSS feeds and aggregate news updates from multiple websites.
Building the News Aggregator App
In this article, we’ll build a news aggregator app using Python (Streamlit), deploy it to Heroku and access it from Heroku via an Android app. Below is the sequence of steps in the app development process.
Before proceeding, we’ll look at how the Android app works in real time on a smartphone. Looking at the app before getting into details may help in a better understanding.
Upon opening the Android app, we see that the browser is connecting to the Streamlit app deployed to Heroku and the app is running. We’ll discuss each step of the app development process in detail, starting with Streamlit app development.
Step 1: Streamlit app development
It is the first and the most important step which does all the processing stuff. We’ll breakdown the to understand it better. Each snippet is explained clearly using comments and doc strings.
- Importing libraries
We’ll import the standard libraries for reading RSS feeds. dateutil package is used to parse dates from string to timestamps.
2. Initialize the list of RSS feeds to capture
Below is a list of RSS feeds to track. I’m interested in stock markets, so I wanted to track the business news from top business news websites in India.
3. Function to calculate the elapsed time
Elapsed time is the difference between the current time and the time when the news article was published. This function computes the time elapsed (in minutes) since the article was published.
4. Function to convert the elapsed time into words
The above function computes the elapsed time in minutes. This function converts the elapsed time in minutes to word form (example: 2 minutes ago, 2 hours ago, etc.).
5. Function to clean the title and description of a news article
There may be cases where few special characters in HTML like " may not be properly parsed. This function replaces such unparsed HTML special characters with characters of the English language.
6. Function to extract the source of the news from RSS feed’s URL
Every news article has a source i.e. the publisher. This function extracts the source (domain name of the news website) from the RSS feed URL.
7. Function to process individual news item of an RSS feed
RSS feeds embed each news article within the <item> tag. This function extracts the required data (title, description, published date, etc.) from the news item and stores it in a data frame.
8. Function to parse each RSS feed
The above function parses each news item (<item>) of an RSS feed. While this function processes each RSS feed by calling the functions declared above and stores the results into a data frame.
9. Snippet to process, store and clean data from all the RSS feeds into a data frame
This step traverses through all the RSS feeds declared as a list and stores it in a final data frame having all the news articles from all the RSS feeds. This step also does some final preprocessing and deletes few unwanted columns from the data frame.
10. Snippet to build the front end of the app
This step builds the frontend HTML script used to display the news articles. A loop traverses through each row of the data frame containing all the news articles and inserts each article into a row of a HTML table.
Step 2: Deploy the Streamlit app to Heroku
Deploying the Streamlit app to ‘Streamlit sharing’ and using it in the Android app isn’t working. Hence, we’ll deploy the Streamlit app to Heroku. The article below clearly demonstrates the process of deploying a Streamlit app to Heroku.
The process of deploying the app to Heroku stays the same as demonstrated in the above article. However, there are a few modifications required to the deployment files which are shown below. setup.sh remains the same as in the above article.
Folder Structure
news_rss.py is the Python script of the Streamlit app built by placing all the snippets of the Step 1: Streamlit app development into it, in the same order.
requirements.txt
streamlit==0.86.0
regex==2021.8.3
lxml==4.6.3
numpy==1.21.1
python-dateutil==2.8.2
pandas==1.3.1
requests==2.26.0
bs4==0.0.1Procfile
web: sh setup.sh && streamlit run news_rss.pyStep 3: Building the Android app
I have no experience with Android app development (hence, this article shouldn’t be considered as an Android app development tutorial). However, I know that it is possible to convert a website into an Android app. As a beginner, there were a few questions raised in my mind. We’ll discuss them one by one.
1. Which IDE do I use to build an Android app?
After Googling, I found many IDEs for android development. I decided to go with the official IDE for Android development, Android Studio.
2. How to start building an app in Android Studio that connects to a website?
After googling, I found a YouTube video and the official documentation demonstrating it. I created the project as an ‘Empty Activity’ and named it as ‘MyNews’.
3. How do I test the app?
I found the official documentation of creating an emulator to test the app. Emulator is a virtual device that replicates an original one. It enables us to test the app without actually installing it on a physical device.
3. Why isn’t my app loading the web page?
I tested the app and found the WebView was not loading the web page. I found the solution in Stackoverflow.
4. How do I add a refresh button?
Once the app is working fine, I wanted to add a refresh button to refresh the page, when intended. Again, I found the solution in Stackoverflow.
5. How do I change the app’s theme?
The default theme of an Android app is purple i.e. the refresh button and the action bar look purple. But, I wanted to change it to black. I found the solution in Stackoverflow. I’ve changed the colors in themes.xml file.
6. How to change the title of the app displayed on the activity bar (banner)?
I found the solution in Stackoverflow.
7. Finally, how do I create the .apk file of my app?
As the app was running fine in the emulator tests, I wanted to create an apk file to install it on my smartphone. I found the way here.
Below are the Android Studio files of the app. These are only the files which I’ve modified (all the other files need not be modified).
activity_main.xml: Specifies the layout of the app (interface)
MainActivity.java: Runs when the app is opened. It also adds actions to the components of the interface.
AndroidManifest.xml: Holds metadata of the app.
themes.xml: Specifies the app’s theme.
strings.xml: Specifies app name, developer info, etc.
This is an example of building a news aggregator Android app for Python developers with no Android development experience. The same process can be followed to convert any Streamlit app into an Android app. This app is only for one’s personal use and doesn’t have personalized content capabilities (which is required to support multiple users). If the RSS feeds are used for commercial purposes, developers must get the consent of the feed’s publishers before proceeding further.
Know more about my work at https://ksvmuralidhar.in/
