How to Convert Github Api Calls to Rss Feeds

Create a React RSS Feed App for YouTube Feeds

Use create-react-app to build your own, personal, YouTube feed.

The Problem

With several hundred YouTube channel subscriptions, it's hard to find the new longer see in your daily feed, making it hard to find new videos for specific categories.

The Solution

This article will walk through creating an app that will have multiple sets of YouTube channel feeds, separated by categories so you can see the latest videos in each category, taking advantage of YouTube RSS (Really Simple Syndication) feeds.

Prerequisites

At the end of this article, there will be a link to a GitHub repository with the completed application. If you want to create the app on your own, you will need a few application installs on your machine.

It's good to have git installed on your machine git Git Bash. With Git Bash, you get a Linux shell for your terminal. This article was written with the following version of git.

          $ git --version ## 2.34.1.windows.1

The target machine I used while writing this article was running on Windows, but you should be able to install these apps and follow along on any operating system. You can download and install git, here.

For JavaScript development, you'll need Node.js with npm and npx installed. For npm and npx, you will need a major version great than 6.0.0. You can install the latest version of Node.js, which will also install npm and npx, here. This article was written in the following versions:

          $ node --version ## v16.13.1
$ npm --version  ## v8.1.2
$ npx --version  ## v8.1.2

Once you have npm installed, you'll want to install create-react-app, using the global install command:

          $ npm install -g create-react-app  ## allows you to use CRA cli
$ create-react-app --version       ## v5.0.1

If you want to explore more of create-react-app's features, the app has great documentation, here.

Another tool you should be using is an IDE. Visual Code is useful and has a good amount of plugins that you can install for JavaScript and React development. If you are interested in using Visual Code, you can download and install it from here.

Getting Started

There will be five steps in creating our app: First, generate the boilerplate application. Second, create the react components we'll be using to display data. Third, we will locate our data source. Fourth, we install and set up our application to interpret the data. And finally, we set up the calls in the application to pull the data and display it in the application.

Generate the App

The first step in creating the app is generating the boiler plate. Go into the directory you keep your project repositories, and in your terminal, generate your fresh CRA.

          $ create-react-app rss-feed-medium-app
$ cd rss-feed-medium-app
$ npm start

This will create and install all the default dependencies, it can take a few minutes. Since everything is installed with this command, you can run the app right after the command finishes running, and see the default content of the CRA program.

Boilerplate Cleanup

The boiler plate is a good start, but we won't need most of the content in the code. So we can go through App.css, App.js, and App.test.js and clear out a bunch of code. Since this coding is almost entirely deleting code, not writing, instead of a code snippet, the git diff shows the updates needed.

On the left in the diff images, the read code is the code removed, and on the right in the green is any additional code added (which is not much yet). First, the biggest change is in the App.css file.

Then, remove code out of the App component in the App.js file.

Finally, update the App.test.js so tests pass when we run it.

You can also remove the src/logo.svg, since that is removed from the App.

Creating View Components

Now we can create the display components for the RSS feed. These components are going to be created with the foresight that we already know the shape of the data for the feed (more details about the data will come).

Setup Component Directory

We will create three view components: an Entry component for each entry of the feed, an EntryList component to display the list of our RSS feed, and a CategorySelector component for picking which type of channels we want from our RSS feed to show up.

First create a components directory under the existing src directory. Then add a directory under components for each component: CategorySelector, Entry, and EntryList. Now in each directory, create two files that match the name of the directory: extensions .js, and .test.js. For the Entry and EntryList component, also add a .css file. Once finished, the src directory will look like this:

The CategorySelector component

The first component we will create, with some tests is the CategorySelector. Here is a GitHub gist of the component with its test.

CategorySelector

To go over the component logic, it receives a list of categories, which we will create later based on the channel types we want to see in our feed. For each category, it creates a radio button. When the button is selected, it emits that selection to a onSelect function to let the parent component know a new group of categories should be shown in the feed.

The CSS file is pretty compact, with just a few attributes:

          /** CategorySelector.css */
.CategorySelector-root input {
            margin-bottom: 12px;
            margin-right: 12px;
}

The RSS Entry component

Now we will want to create the Entry component for the individual feed entries. The component will have some basic data points, with a thumbnail image (provided from a URL), a JSON date string, and a link to open the video in a new browser tab.

The style sheet for the Entry component has most of the styles in the app since it arranges the data points and the image. Here is the gist for the styles.

Ultimately, each RSS entry will appear like this image (minus the lorem ipsum text, compliments of hipsum.co).

The EntryList component

The last component to make is the EntryList component to display all the RSS entries. It's entire purpose is to take in a list of RSS entries, and iterate over them to display.

Since the point of EntryList is to iterate over the RSS data, it does not have a CSS file.

Adding Components to App

The last step we do before beginning to extract and set up the data is to add the components to the App component.

You can see in the gist that we added useState for maintaining a list of RSS entry objects, and another list of category strings.

There is now a useEffect setup with an empty dependency array. It will be used to generate a list of categories based off static data that will be added with the list of RSS feed URLs. The list of categories will be passed into the CategorySelector component categories prop.

The handler function passed into the CategorySelector, onSelect prop is setup to eventually load the data for the entries prop that feeds the EntryList.

Setup Data Source

The next few steps will be establishing our data sources, setting up some static data, and exploring how to massage the data.

YouTube RSS feed URLs

Recently, YouTube has made finding channel RSS feeds somewhat obfuscated. In order to find a channels feed URL, you'll need to go to that channel's homepage. Then, in a Chrome browser, right click and select "View Page Source" (on Windows you could also click Ctrl + U).

Now, once the page source is open, you can do a "Find", and search the page for the RSS feed URL. To do this, open the browser's controls (the three dots in the upper right corner) and click "Find", or press Ctrl + F on windows. Search for title="RSS", and press Enter. You will be brought to a link tag with the RSS feed metadata for the channel.

Grab the href attribute value and Copy it. If you paste it into a new browser tab, you can see the XML payload that will come back.

Looking at the payload in the browser, you can see that there is a lot of metadata that can be used. There are a total of 15 entry sections that come down. That is the max number that will come back, and it is the 15 latest releases from the channel.

Loading Feeds into the App

Now we can create a lib.js file in the src directory. In this lib, there will be a fetchFeed function that, for now, takes no arguments and has the channel RSS feed URL hard coded.

We'll use the fetch function to make the API call since this request is a simple GET. Along with the feed URL, we'll also need to use a CORS proxy. This is because YouTube doesn't allow requests that originate from localhost.

After the data is fetched, invoke text() on the response to get a string of the XML. For now, all the callbacks will simply log content to the console to check what is happening in the browser.

          const corsProxy = "https://cors.eu.org/";
const fetchFeed = () => {
            const url = "https://www.rss-feed-url.com";
            fetch(corsProxy + url)
            .then((res) => res.text()
            .then((data) => console.info("data", data))
            .catch((err) => console.warn("data err", err))
            .finally(() => console.info("data complete")))
            .catch((err) => console.warn("res err", err))
            .finally(() => console.info("res complete"));
};          export { fetchFeed };

To test out the function, we can add it to the useEffect in the App component with nothing in the dependency array. Here is the console of the browser after invoking the fetchFeed() function in the app.

Upgrading the Application

In order to process the XML from the feeds, we'll need to configure the project to use some new packages.

Installing new dependencies

First, we will install the required dependencies here is the list:

          $ npm i xml2js             ## for processing the XML to JSON
$ npm i buffer             ## xml2js dependency
$ npm i process            ## xml2js dependency
$ npm i stream-browserify  ## xml2js dependency
$ npm i timers-browserify  ## xml2js dependency
$ npm i react-app-rewired  ## for webpack configuration overriding

The xml2js package is used to convert XML to JSON, with the four following packages needed in order to allow xml2js to run. The last dependency, react-app-rewired, is used to configure webpack, the tool that compiles our JavaScript into a single file, inside a create-react-app, without ejecting the configurations.

Rewiring the App

Now we can set up the override file.

          // config-overrides.js
const { ProvidePlugin } = require("webpack");          module.exports = function override(config) {
            config.plugins.push(new ProvidePlugin({
            Buffer: ["buffer", "Buffer"]
            }));
            config.plugins.push(new ProvidePlugin({
            process: "process/browser"
            }));
            config.resolve.fallback = {
            buffer: require.resolve("buffer/"),
            stream: require.resolve("stream-browserify"),
            timers: require.resolve("timers-browserify"),
            };
            return config;
};

Setup react-app-rewired with config-overrides.js
We'll have to also update the scripts section in the package.json to pickup the rewire. Here is how the changes look, with the original scripts on the left in red, and the new scripts on the right in green.

Pulling the Data

Now that the app is set up to consume the XML as JSON, we can update the fetchFeed function.

Parsing XML

First, import the parseString function from the xml2js package into the lib.js file.

Setup processing for XML to JSON. The function takes in two parameters: an XML string and a callback function. The callback signature is function callback(error, result);.

The error is only defined when an error occurs when parsing the string, and the result is the JSON conversion of the XML. We can see how:

          import { parseString } from "xml2js"          // original fetch() call
            fetch(corsProxy + url)
            .then((res) => res.text()
            .then((data) => console.info("data", data))
            .catch((err) => ...          // updated fetch() call
            fetch(corsProxy + url)
            .then((res) => res.text()
            .then((data) => parseString(data, (err, res) => {
            console.warn("err", err);
            console.info("res", res);
            }))
            .catch((err) => ...

After updating the function and running the App, we can see in the console of the browser how the result object is shaped. Explore it, and see that all the RSS entry data is stored in an array, under the key result.feed.entry.

We only want pieces of this payload, so we can process the data here and shape how we want the App to use it. The only values the app need are: an author, a video name, a link to the video, a publish date, a link to a thumbnail image, and a unique Id.

Exploring the shape in the browser console to find all these attributes in the entries will show us how to pull that data from the payload. We can write a callback function for parseString that pulls all this data.

Now, instead of just logging the cleaned data to the console, we want to pull this data into the application. We can do this with a callback function.

Loading Data into App

First, we will create a static feeds object that will be used to create a categoryList, as well as store the URLs for channel feeds. To do this, we will create a new file and directory under src, which will be assets/feeds.js. This file is just going to contain an object, with a list of categories as keys to an array of the URL string, like this:

          // src/assets/feeds.js
const feeds = {
            "comedy": [
            // channel name here
            "https://www.comedy-channel.com/rss/feed",
            ],
            "gaming": [
            // channel name here
            "https://www.gaming-channel.com/rss/feed",
            ],
            "techno": [
            // channel name here
            "https://www.techno-music-channel.com/rss/feed",
            ],
};          export default feeds;

Then, import the feeds object into the lib.js file, and set up a new function that will accept a category name and a callback: fetchCategroyFeeds. Update the fetchFeed function to accept two args as well; category and a callback.

The parseStringCallback now also takes in a callback argument. This callback is used to return the processed feed data back to the component. The lib.js file will look like this new version.

In the App.js file, add the feeds to the component and use if to create a categoryList, using the function Object.keys(feeds). Set the categories state to the categoryList in the first useEffect.

Then the handleSelect function can be updated to use the fetchCategoryFeeds, and pass in the selectedCategory argument as the first parameter, and then pass the setEntries state call as the callback. This will load the retrieved feeds as the state.

At this point, you should be pulling in the data and displaying the entries depending on which radio button you select at the top.

Merging Feed Data

The final step is to set up multiple feeds in each category. Once you find multiple feeds and pull their RSS feeds, your feeds.js file will look like this:

          // src/assets/feeds.js
const feeds = {
            "comedy": [
            // channel one name here
            "https://www.comedy-one-channel.com/rss/feed",
            // channel two name here
            "https://www.comedy-two-channel.com/rss/feed",
            ],
            "gaming": [
            // channel one name here
            "https://www.gaming-one-channel.com/rss/feed",
            // channel two name here
            "https://www.gaming-two-channel.com/rss/feed",
            // channel three name here
            "https://www.gaming-three-channel.com/rss/feed",
            ],
            "techno": [
            // channel one name here
            "https://www.techno-one-music-channel.com/rss/feed",
            // channel two name here
            "https://www.techno-two-music-channel.com/rss/feed",
            ],
};          export default feeds;

You can create a merge function in the App.js component in order to make calls to multiple feeds, then sort the entries from each feed by publish date. You will also replace the callback param with the new handleEntriesMerge method instead of the setEntries.

          const handleSelect = (selectedCategory) => {
            setEntries([]);
            fetchCategoryFeeds(selectedCategory, handleEntriesMerge);
}          const handleEntriesMerge = (data) => {
            setEntries((prev) => {
            return prev.concat(data).sort((a, b) => {
            const dateA = new Date(a.publishDate);
            const dateB = new Date(b.publishDate);
            return dateA > dateB ? -1 : 1;
            });
            });
}

Then in the lib.js file, update the fetchCategoryFeeds to loop through all the feed URLs.

          const fetchCategoryFeeds = (category, callback) => {
            const { length } = feeds[category];
            for (let idx = 0; idx < length; idx++) {
            fetchFeed(feeds[category][idx], callback);
            }
};

The Final Product

Here is what the finished product looks like with multiple feeds in each category.

Here is the final source code on GitHub with all the changes to create the application. You can check out the exact version here with the command:

          $ git clone https://github.com/cameronDz/rss-feed-medium-app.git
$ cd rss-feed-medium-app
$ git checkout tags/v1.0.0
$ npm install
$ npm run start

hardingkessad.blogspot.com

Source: https://javascript.plainenglish.io/create-a-react-rss-feed-app-for-youtube-feeds-69c8cd2dbd46