Get notified when a file on GitHub has changed

Waldek Mastykarz - Dec 22 '20 - - Dev Community

Get notified when a file on GitHub has changed

With more and more content being hosted on GitHub it's a shame that you can't get notified when a particular file has changed. At least not by GitHub, that is. But you can build your own notification service. Here is how.

Adaptive card with a notification about a changed file on GitHub displayed in Microsoft Teams

More than source control

GitHub is more than just the place where developers post their code. Over the last few years, it evolved also into a huge repository of content published through blogs and docs. For many, it's the go-to place to stay up-to-date on what's new.

While GitHub offers us a way to get notified of changes to repos, the notifications options are limited. You can get notified of new releases or pull requests, but you can't get notified when a particular file has changed. With many devs using specific pages for announcing releases or keeping their changelog, being able to get notified of changes to these specific files, allows you to stay up-to-date on what's new and what's coming.

Build your own GitHub file changes notifier

To build your own GitHub file changes notifier, all you need is an Azure subscription. In the subscription, you will create a function app, a storage account and a logic app. Here is the high-level of the solution architecture:

Diagram showing architecture of the solution

The storage account holds the list of files to check for changes. For each file, you will store its URL, title - for use with notifications, and a hash of its contents used to determine if the file has changed since the last time.

The function app hosts the scheduled process that at the specified time retrieves the list of files from the storage account (1), iterates through them, for each (2) one checks if it changed and sends a notification if it did (3). If the file has changed, the function app also updates the file hash in the storage account (4).

While there are unlimited ways to send a notification, I find using logic apps the easiest. Logic apps are extremely versatile and with just a few clicks you can set them up to receive notifications on Teams, via email, or on your phone, without having to deal with authentication, response handling, etc.

Set up a storage account

Start, by creating a storage account in your resource group. In the storage account, create a table named files. For each file that you want to track, insert an item to the table including its title and url.

Table row showing information about a file on GitHub to monitor for changes

For each file, use the same partition key and a unique row key.

Create a logic app for sending notifications

In this example, we'll create a logic app that sends a notification to the user on Microsoft Teams.

In your resource group, create a logic app. Use the When a HTTP request is received trigger. As a sample payload, to generate the request schema specify:

{
  "title": "abc",
  "url": "def"
}
Enter fullscreen mode Exit fullscreen mode

Then, add the Post your own adaptive card as the Flow bot to a user action. Specify your email as the recipient. Choose two parameters: Message and Headline. In the Message parameter, enter:

{
  "type": "AdaptiveCard",
  "body": [
    {
      "type": "TextBlock",
      "size": "Medium",
      "weight": "Bolder",
      "text": "File changed"
    },
    {
      "type": "TextBlock",
      "text": "File '@{triggerBody()?['title']}' has changed",
      "wrap": true
    }
  ],
  "actions": [
    {
      "type": "Action.OpenUrl",
      "title": "View",
      "url": "@{triggerBody()?['url']}"
    }
  ],
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "version": "1.2"
}
Enter fullscreen mode Exit fullscreen mode

In the Headline parameter enter:

File '@{triggerBody()?['title']}' has changed
Enter fullscreen mode Exit fullscreen mode

Save the logic app, and from the trigger, copy the URL which you will need to call to trigger the notification.

Set up function app

Next, create a function app running Node.js. In the app, create a new Timer-triggered function. Adjust the schedule so that the function runs as frequently as you want to check for changes in files.

Create function app panel in the Azure portal

Add Azure Table Storage binding

In your function's integration options, add a new Azure Table Storage input binding pointing to your storage account and the file table. In the partition key property, specify the name of the partition key that you used when entering files to track in the table. Take note of the Table parameter name and the Storage account connection values as you will need them later, in your code.

Store logic app trigger URL

Next, in your function's configuration settings, add a new application setting named NotificationUrl and use the previously copied logic app trigger URL as its value.

Add function's code

Then, add the function's code:

const axios = require('axios').default;
const crypto = require('crypto');

module.exports = async function (context, timer, inFilesTable) {
  const timeStamp = new Date().toISOString();
  context.log('Checking for file changes...', timeStamp, context.invocationId);

  // array of files that have changed
  const modifiedFiles = [];
  // array of notifications
  const modifiedFilesNotifications = [];

  // retrieve file contents
  const requests = inFilesTable.map(file => {
    // if it's a github URL, make it point to the raw file to avoid comparing github UI changes
    file.rawUrl = file.url
      .replace('//github.com/', '//raw.githubusercontent.com/')
      .replace('/blob/', '/');
    return axios.get(file.rawUrl);
  });
  const results = await Promise.allSettled(requests);

  results.forEach(result => {
    if (result.status !== 'fulfilled') {
      context.log.error(result.reason.message, result.reason.config.url, context.invocationId);
      return;
    }

    const requestedUrlPath = result.value.request.path;
    const matchingFile = inPagesTable.find(page => page.rawUrl.indexOf(requestedUrlPath) > -1);
    if (!matchingFile) {
        context.log.error('Matching file not found in database', requestedUrlPath, context.invocationId);
        return;
    }

    const hash = crypto.createHash('sha512');
    hash.update(result.value.data);
    const currentHash = hash.digest('hex');

    context.log.verbose('Comparing file hash', requestedUrlPath, context.invocationId)
    context.log.verbose('Old hash', matchingFile.hash, context.invocationId);
    context.log.verbose('Current hash', currentHash, context.invocationId);
    if (matchingFile.hash !== currentHash) {
      if (typeof matchingFile.hash !== 'undefined') {
        context.log('File has changed', matchingFile.url, context.invocationId);
        modifiedFilesNotifications.push(matchingFile);
      }
      else {
        context.log('Initial hash retrieved', matchingFile.url, context.invocationId);
      }

      matchingFile.hash = currentHash;
      modifiedFiles.push(matchingFile);
    }
    else {
      context.log('File not changed', matchingFile.url, context.invocationId);
    }
  });

  // send notifications for changed files
  const notificationUrl = process.env.NotificationUrl;
  const notificationRequests = modifiedFilesNotifications.map(file => axios.post(notificationUrl, {
    title: file.title,
    url: file.url
  }));

  const notificationResults = await Promise.allSettled(notificationRequests);
  notificationResults.forEach(result => {
    if (result.status !== 'fulfilled') {
      let fileChangedUrl;
      try {
        fileChangedUrl = JSON.parse(result.reason.config.data).url;
      }
      catch { }
      context.log.error(result.reason.message, result.reason.config.url, fileChangedUrl, context.invocationId);
      return;
    }
  });

  // update modified files in the database
  if (modifiedFiles.length > 0) {
    const azure = require('azure-storage');
    const tableSvc = azure.createTableService(process.env.filetracker_STORAGE);
    const batch = new azure.TableBatch();
    modifiedFiles.forEach(file => {
      batch.mergeEntity({
        PartitionKey: { _: file.PartitionKey },
        RowKey: { _: file.RowKey },
        hash: { _: file.hash }
      }, { echoContent: true });
    });
    tableSvc.executeBatch('files', batch, function (error, result, response) {
      if (error) {
        context.log.error(error);
      }

      context.log.verbose(result);
    });
  }

  context.done();
};
Enter fullscreen mode Exit fullscreen mode

Let's walk through the different pieces of the script.

Load dependencies

Start, by loading the necessary dependencies:

const axios = require('axios').default;
const crypto = require('crypto');
Enter fullscreen mode Exit fullscreen mode

We'll use axios to execute web requests to download files and trigger notifications. axios makes it way easier than using the raw HTTP calls from Node.js. We also need the crypto functions so that we can calculate the hash for the downloaded file contents.

Reference input binding

Extend the function's function to include the Table parameter name you copied previously. This argument will contain all files that you've added to the storage account.

module.exports = async function (context, timer, inFilesTable) {
Enter fullscreen mode Exit fullscreen mode

Store information about changed files

Next, let's define two arrays for storing information about changed files:

// array of files that have changed
const modifiedFiles = [];
// array of notifications
const modifiedFilesNotifications = [];
Enter fullscreen mode Exit fullscreen mode

We need two arrays because if we've just added a file to the storage account, its hash (null) will not match the hash retrieved from GitHub and would result in you getting a notification that the file has changed, while it's not quite the case. Storing information about the file that changed and files for which we should issue a notification allows avoiding the unnecessary initial notification.

Retrieve file contents

Then, let's retrieve contents of all files we're tracking:

// retrieve file contents
const requests = inFilesTable.map(file => {
  // if it's a github URL, make it point to the raw file to avoid comparing github UI changes
  file.rawUrl = file.url
    .replace('//github.com/', '//raw.githubusercontent.com/')
    .replace('/blob/', '/');
  return axios.get(file.rawUrl);
});
const results = await Promise.allSettled(requests);
Enter fullscreen mode Exit fullscreen mode

Before we configure the web requests, we rewrite the GitHub URL. That way, we'll download the raw file contents and won't issue change notifications if the GitHub UI changes.

We download file contents in parallel using the Promise.allSettled function. That way, even if we can't download some files (if they're for example renamed, removed or there was a network error), the files that we could download will be processed without failing the whole execution.

After all promises have been completed, we'll iterate through the results:

results.forEach(result => {
  // ...
}
Enter fullscreen mode Exit fullscreen mode

Logging failed requests

The first thing we need to do is to check if the particular request succeeded or not.

if (result.status !== 'fulfilled') {
  context.log.error(result.reason.message, result.reason.config.url, context.invocationId);
  return;
}
Enter fullscreen mode Exit fullscreen mode

We do this by examining the status property. If the particular request failed, we log it, so that afterward we can see that something went wrong and we have sufficient information to determine if it was something intermittent or rather the URL is no longer valid.

Retrieve matching file

Next, let's retrieve the information about the file matching the current response. We need it to compare the new file hash as well as for passing the information to notifications we send in case the file has been modified.

const requestedUrlPath = result.value.request.path;
const matchingFile = inPagesTable.find(page => page.rawUrl.indexOf(requestedUrlPath) > -1);
if (!matchingFile) {
    context.log.error('Matching file not found in database', requestedUrlPath, context.invocationId);
    return;
}
Enter fullscreen mode Exit fullscreen mode

We need to look up this information to avoid relying on the order in which promises will be resolved.

Check if the file has been modified

Now that we have contents for each of our tracked files, let's calculate its hash and compare it to the one we retrieved previously:

const hash = crypto.createHash('sha512');
hash.update(result.value.data);
const currentHash = hash.digest('hex');

context.log.verbose('Comparing file hash', requestedUrlPath, context.invocationId)
context.log.verbose('Old hash', matchingFile.hash, context.invocationId);
context.log.verbose('Current hash', currentHash, context.invocationId);
if (matchingFile.hash !== currentHash) {
  if (typeof matchingFile.hash !== 'undefined') {
    context.log('File has changed', matchingFile.url, context.invocationId);
    modifiedFilesNotifications.push(matchingFile);
  }
  else {
    context.log('Initial hash retrieved', matchingFile.url, context.invocationId);
  }

  matchingFile.hash = currentHash;
  modifiedFiles.push(matchingFile);
}
else {
  context.log('File not changed', matchingFile.url, context.invocationId);
}
Enter fullscreen mode Exit fullscreen mode

If the file hash is different from the one we stored previously, we update its hash and add it to the array of files that should be updated in the database. Additionally, if the hash has been set previously, we add the file to the array of notifications.

Notify of file changes

Once we processed all files, we're ready to send notifications to our previously configured logic app.

// send notifications for changed files
const notificationUrl = process.env.NotificationUrl;
const notificationRequests = modifiedFilesNotifications.map(file => axios.post(notificationUrl, {
  title: file.title,
  url: file.url
}));

const notificationResults = await Promise.allSettled(notificationRequests);
notificationResults.forEach(result => {
  if (result.status !== 'fulfilled') {
    let fileChangedUrl;
    try {
      fileChangedUrl = JSON.parse(result.reason.config.data).url;
    }
    catch { }
    context.log.error(result.reason.message, result.reason.config.url, fileChangedUrl, context.invocationId);
    return;
  }
});
Enter fullscreen mode Exit fullscreen mode

We start with retrieving the URL of the logic app from the NotificationUrl environment variable that's populated by the application configuration entry we added when creating the function app.

Next, we create a POST request matching the schema configured in the logic app.

Finally, we issue all requests, again, not breaking the execution in case any of the requests failed. Instead, we log failed requests for further investigation.

Update hashes of modified files

The last thing left is to update the hashes of modified files. Unfortunately, the output Azure Table Storage binding doesn't support updating entities, so for that, we'll need to update it using the Azure Storage SDK.

// update modified files in the database
if (modifiedFiles.length > 0) {
  const azure = require('azure-storage');
  const tableSvc = azure.createTableService(process.env.filetracker_STORAGE);
  const batch = new azure.TableBatch();
  modifiedFiles.forEach(file => {
    batch.mergeEntity({
      PartitionKey: { _: file.PartitionKey },
      RowKey: { _: file.RowKey },
      hash: { _: file.hash }
    }, { echoContent: true });
  });
  tableSvc.executeBatch('files', batch, function (error, result, response) {
    if (error) {
      context.log.error(error);
    }

    context.log.verbose(result);
  });
}
Enter fullscreen mode Exit fullscreen mode

We start by loading the SDK. We do this here since it's a pretty expensive operation, that would unnecessarily slow down the function's execution. Lazy-loading the SDK only when it's needed, will help you speed up the function.

Next, we create an instance of table service, passing the connection string to the table storage with our files. The name of the Storage account connection you noted earlier when configuring the input binding in your function app.

Then we create an update batch so that we can update all entities in a single request, saving us some time. For each file to update, we specify its partition key, row key, and the new hash.

We finish, by executing the batch and logging errors if any.

Install dependencies

While the function's code is complete, you need to install dependencies, before you will be able to run it. In our code, we referenced axios and azure-storage which we need to install in the function for the code to run without errors.

To install dependencies, from the function's menu, select the App Service Editor.

The 'App Service Editor' option highlighted in the Azure portal

In the App Service Editor, from the side menu, select the Open Console option.

The 'Open Console' option highlighted in the App Service Editor

In the console execute:

npm i axios azure-storage
Enter fullscreen mode Exit fullscreen mode

After the installation is completed, you're ready to test your function.

Summary

That's it! With a couple of steps, you've built a scheduled process that runs in the cloud, monitors specified files for changes, and notifies you when they do on Microsoft Teams.

Adaptive card with a notification about a changed file on GitHub displayed in Microsoft Teams

Image
This Is The One Internal Injury Lawyer Trick Every Person Should Learn
Image
Building a Self-Evolving Codebase: JavaScript’s AST Manipulation and Dynamic Compilation

javascript, webdev, node, programming
Image
Why Learn Rust Programming Language?

webdev, rust, tutorial, learninhindi
Image
The Lesser-Known Benefits Of Asbestos Attorneys Near Me
Image
First Blog

webdev, beginners, bootcamp
Image
macam keuntungan main di slots online
Image
20 Questions You Need To To Ask About Head Injury Settlement Amount Before Purchasing It
Image
Do Not Forget Mental Health Test: 10 Reasons Why You Don't Need It
Image
Private Mental Health Care Tools To Enhance Your Day-To-Day Life
Image
Tehnik Jadi Pemain Judi Online Professional
Image
Stwórz Swoją Idealną Sypialnię: Meble, Które Odmienią Twoją Przestrzeń
Image
7 Simple Tricks To Moving Your Mental Health Therapist Near Me
Image
Top 10 Docker Commands Every Developer Should Know [Chapter 1]

docker, devops, cloud, kubernetes
Image
Menunjuk Agen Judi Online Jekpot Berlimpah
Image
How to Clean a Single Gas Oven
Image
10 Easy Steps To Start The Business You Want To Start Coffee Pot Drip Business
Image
Best Built in Single Oven
Image
The Reasons You'll Want To Find Out More About Private Mental Health Assessment
Image
Five Tools That Everyone Involved In Mental Health Specialist Near Me Industry Should Be Using
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player