Monitoring GitHub for new releases#
Big sites like GitHub or GitLab are hosting a lot of projects and have numerous of releases a day. And while you as a person can watch a repository on GitHub, you can’t filter out new releases easily. At least not easily findable in the interfaces and checking all the repositories manually because they aren’t part of a build process is too much hassle and will fail in the end. So also for me with highlight.js as it has been updated from version 9.11.0 to 9.12.0 months ago.
Looking at some solutions people were writing about on StackOverflow for example was to parse the HTML and use that as a basis for actions to be executed. A quick check and grep of the output shown that we only have links to releases, but no structured data we can easily parse.
$ curl -s https://github.com/isagalaev/highlight.js/releases | grep -i releases\/tag
<a href="/isagalaev/highlight.js/releases/tag/9.12.0">
<a href="/isagalaev/highlight.js/releases/tag/9.11.0">
<a href="/isagalaev/highlight.js/releases/tag/9.10.0">
<a href="/isagalaev/highlight.js/releases/tag/9.9.0">
<a href="/isagalaev/highlight.js/releases/tag/9.8.0">
<a href="/isagalaev/highlight.js/releases/tag/9.7.0">
<a href="/isagalaev/highlight.js/releases/tag/9.6.0">
<a href="/isagalaev/highlight.js/releases/tag/9.5.0">
<a href="/isagalaev/highlight.js/releases/tag/9.4.0">
<a href="/isagalaev/highlight.js/releases/tag/9.3.0">
If we take the same URL and add the extension .atom to it, then GitHub presents the same data in a consumable feed format. Now we have structured data with timestamps, URLs, and descriptions.
$ curl -s https://github.com/isagalaev/highlight.js/releases.atom
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" xml:lang="en-US">
<id>tag:github.com,2008:https://github.com/isagalaev/highlight.js/releases</id>
<link type="text/html" rel="alternate" href="https://github.com/isagalaev/highlight.js/releases"/>
<link type="application/atom+xml" rel="self" href="https://github.com/isagalaev/highlight.js/releases.atom"/>
<title>Release notes from highlight.js</title>
<updated>2017-05-31T02:46:46Z</updated>
<entry>
<id>tag:github.com,2008:Repository/1213225/9.12.0</id>
<updated>2017-05-31T02:46:46Z</updated>
<link rel="alternate" type="text/html" href="/isagalaev/highlight.js/releases/tag/9.12.0"/>
<title>9.12.0</title>
<content type="html"><p>Version 9.12.0</p></content>
<author>
<name>isagalaev</name>
</author>
<media:thumbnail height="30" width="30" url="https://avatars2.githubusercontent.com/u/99931?v=4&s=60"/>
</entry>
...
This data can be used by a custom parser or RSS-readers like TT-RSS, but also used by platforms like IFTTT to trigger actions like adding it to a backlog or posting it to a Slack-channel.