https://dailystuff.nlDailystuff on the Internet - Posts in Web Development2024-03-16T09:59:02.407571+00:00ABloghttps://dailystuff.nl/blog/2023/adding-google-analytics-in-sphinx.htmlAdding Google Analytics to Sphinx2023-11-24T00:00:00+00:00Hans Spaans<section id="adding-google-analytics-to-sphinx">
<p>Creating and maintaining a website is a lot of work and it is nice to know which pages are being visited and how people are finding your site. <a class="reference external" href="//analytics.google.com/">Google Analytics</a> is a great tool for this and it is easy to add to your Sphinx site. But adding it to your <a class="reference external" href="https://github.com/sphinx-doc/sphinx">Sphinx</a> Sphinx blog is not as easy as it could be. This post will show you how to add Google Analytics to your Sphinx site instead of modifiying the theme or template files.</p>
<section id="preparing-your-site">
<h2>Preparing your site</h2>
<p>The first step is to create a Google Analytics account and get your tracking ID. This is a long string that looks like <code class="docutils literal notranslate"><span class="pre">G-XXXXXXXXX</span></code>. You can find it in the admin section of your Google Analytics account.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If you are using Google Analytics 4, you will need to use the <code class="docutils literal notranslate"><span class="pre">G-XXXXXXXXX</span></code> format. If you are using Universal Analytics, you will need to use the <code class="docutils literal notranslate"><span class="pre">UA-XXXXXXXXX</span></code> format.</p>
</div>
<p>The next step is to install the <a class="reference external" href="https://github.com/sphinx-contrib/googleanalytics">sphinxcontrib-googleanalytics</a> package. This is a Sphinx extension that will add the Google Analytics tracking code to your site. You can install it with <code class="docutils literal notranslate"><span class="pre">pip</span></code> or add it to your <code class="docutils literal notranslate"><span class="pre">requirements.txt</span></code> file.</p>
<div class="literal-block-wrapper docutils container" id="id1">
<div class="code-block-caption"><span class="caption-text">Adding the <code class="docutils literal notranslate"><span class="pre">sphinxcontrib-googleanalytics</span></code> package to <code class="file docutils literal notranslate"><span class="pre">docs/requirements.txt</span></code></span></div>
<div class="highlight-toml notranslate"><div class="highlight"><pre><span></span><span class="p">...</span>
<span class="n">sphinxcontrib-googleanalytics</span><span class="o">=</span><span class="err">=</span><span class="mf">0.4</span>
</pre></div>
</div>
</div>
</section>
<section id="enabling-the-extension">
<h2>Enabling the extension</h2>
<p>Now that you have the extension installed, you need to enable it in your Sphinx configuration file. You can do this by adding <code class="docutils literal notranslate"><span class="pre">sphinxcontrib.googleanalytics</span></code> to the <code class="docutils literal notranslate"><span class="pre">extensions</span></code> list and setting the <code class="docutils literal notranslate"><span class="pre">googleanalytics_id</span></code> variable to your tracking ID.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text">Example configuration file <code class="file docutils literal notranslate"><span class="pre">docs/conf.py</span></code></span></div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">extensions</span> <span class="o">=</span> <span class="p">[</span>
<span class="o">...</span>
<span class="s2">"sphinxcontrib.googleanalytics"</span>
<span class="p">]</span>
<span class="n">googleanalytics_id</span> <span class="o">=</span> <span class="s2">"G-XXXXXXXXX"</span>
</pre></div>
</div>
</div>
<p>After you have added the extension and set the tracking ID, you can build your site and the tracking code will be added to the pages. You can verify this by looking at the source of your pages and searching for the tracking ID.</p>
</section>
<section id="enabling-the-extension-only-when-building-on-github-actions">
<h2>Enabling the extension only when building on GitHub Actions</h2>
<p>The extension will add the tracking code to your pages when you build your site locally. But you probably don’t want to track your local builds. You can use the <code class="docutils literal notranslate"><span class="pre">os.getenv</span></code> function to only enable the extension when you are building on <a class="reference external" href="//docs.github.com/actions/">GitHub Actions</a>.</p>
<div class="literal-block-wrapper docutils container" id="id3">
<div class="code-block-caption"><span class="caption-text">Example configuration file <code class="file docutils literal notranslate"><span class="pre">docs/conf.py</span></code></span></div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"GITHUB_ACTIONS"</span><span class="p">):</span>
<span class="n">extensions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"sphinxcontrib.googleanalytics"</span><span class="p">)</span>
<span class="n">googleanalytics_id</span> <span class="o">=</span> <span class="s2">"G-XXXXXXXXX"</span>
</pre></div>
</div>
</div>
<p>Now the extension will only be enabled when you build your site on GitHub Actions. You can verify this by looking at the source of your pages and searching for the tracking ID.</p>
<p>The example below shows how to enable the extension when building on GitHub Actions, Travis CI, CircleCI, or GitLab CI.</p>
<div class="literal-block-wrapper docutils container" id="id4">
<div class="code-block-caption"><span class="caption-text">Example configuration file <code class="file docutils literal notranslate"><span class="pre">docs/conf.py</span></code></span></div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"GITHUB_ACTIONS"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"TRAVIS"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"CIRCLECI"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"GITLAB_CI"</span><span class="p">):</span>
<span class="n">extensions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"sphinxcontrib.googleanalytics"</span><span class="p">)</span>
<span class="n">googleanalytics_id</span> <span class="o">=</span> <span class="s2">"G-XXXXXXXXX"</span>
</pre></div>
</div>
</div>
</section>
</section>
Creating and maintaining a website is a lot of work and it is nice to know which pages are being visited and how people are finding your site. Google Analytics is a great tool for this and it is easy to add to your Sphinx site. But adding it to your Sphinx Sphinx blog is not as easy as it could be. This post will show you how to add Google Analytics to your Sphinx site instead of modifiying the theme or template files.The first step is to create a Google Analytics account and get your tracking ID. This is a long string that looks like G-XXXXXXXXX. You can find it in the admin section of your Google Analytics account.2023-11-24T00:00:00+00:00https://dailystuff.nl/blog/2023/using-robots.txt-with-cloudflare-pages.htmlUsing robots.txt with Cloudflare Pages2023-07-02T00:00:00+00:00Hans Spaans<section id="using-robots-txt-with-cloudflare-pages">
<p>Cloudflare Pages is a great way to host your static website. It’s fast, easy, and free to start with. While the egress traffic is free, you still pay in other forms as some bots will crawl your website. This is where robots.txt comes in handy. You can use it to block bots from crawling your website without having to configure anything with in Cloudflare.</p>
<section id="use-robots-txt-for-a-sitemap">
<h2>Use robots.txt for a sitemap</h2>
<p>The file <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> is used to tell bots where to find your sitemap. This is useful for search engines to find your sitemap. The sitemap is used to tell search engines what pages are available on your website. Lets start with a simple example to tell bots where to find your sitemap.</p>
<div class="literal-block-wrapper docutils container" id="id1">
<div class="code-block-caption"><span class="caption-text">Exampe file <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> with a sitemap reference</span></div>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>sitemap: https://dailystuff.nl/sitemap.xml
</pre></div>
</div>
</div>
<p>After uploading the file together with the sitemap to the website, you can check if it’s working by visiting <code class="file docutils literal notranslate"><span class="pre">https://dailystuff.nl/robots.txt</span></code>.</p>
</section>
<section id="using-robots-txt-to-block-bots">
<h2>Using robots.txt to block bots</h2>
<p>The file <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> can also be used to block bots from crawling your website. This is useful for bots that are not following the rules set in <span class="target" id="index-0"></span><a class="rfc reference external" href="https://datatracker.ietf.org/doc/html/rfc9309.html"><strong>RFC 9309</strong></a>. For example, the bot is crawling your website too fast or is using too much bandwidth. You can block these bots for the whole site, due to the <code class="docutils literal notranslate"><span class="pre">Disallow:</span> <span class="pre">/</span></code> statement, by adding the following to your <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> file.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text"><code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code></span></div>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>sitemap: https://dailystuff.nl/sitemap.xml
User-agent: Nuclei
User-agent: WikiDo
User-agent: Riddler
User-agent: PetalBot
User-agent: Zoominfobot
User-agent: Go-http-client
User-agent: Node/simplecrawler
User-agent: CazoodleBot
User-agent: dotbot/1.0
User-agent: Gigabot
User-agent: Barkrowler
User-agent: BLEXBot
User-agent: magpie-crawler
User-agent: MJ12bot
User-agent: AhrefsBot
Disallow: /
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The list of bots is not complete and have been randomly selected for the example. You can find a more complete list at <a class="reference external" href="https://www.robotstxt.org/db.html">robotstxt.org</a>.</p>
</div>
</section>
<section id="block-cloudflare-endpoints-via-robots-txt">
<h2>Block Cloudflare endpoints via robots.txt</h2>
<p>Another use case for the <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> file is to block Cloudflare endpoints. These endpoints are used by Cloudflare to provide services like the Web Application Firewall (WAF) and the Bot Management. You can block these endpoints by adding the following to your <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> file and uploading it to your website.</p>
<div class="literal-block-wrapper docutils container" id="id3">
<div class="code-block-caption"><span class="caption-text"><code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code></span></div>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>sitemap: https://dailystuff.nl/sitemap.xml
User-agent: *
Disallow: /cdn-cgi/bm/cv/
Disallow: /cdn-cgi/challenge-platform/
Disallow: /cdn-cgi/images/trace/
Disallow: /cdn-cgi/rum
Disallow: /cdn-cgi/scripts/
Disallow: /cdn-cgi/styles/
Disallow: /cdn-fpw/sxg/
</pre></div>
</div>
</div>
<p>After this <a class="reference external" href="https://search.google.com/search-console/about">Google Search Console</a> will show you a warning that the robots.txt file is blocking Google from crawling your website. This is expected and you can ignore this warning or remove the endpoints from the <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> file, but then the endpoints will be crawled by bots and be part of the crawl budget.</p>
</section>
<section id="block-cloudflare-endpoints-via-http-headers">
<h2>Block Cloudflare endpoints via HTTP headers</h2>
<p>Another option to block Cloudflare endpoints is to use HTTP headers. This is useful if you don’t want to use the <code class="file docutils literal notranslate"><span class="pre">robots.txt</span></code> file. You can add the following to your <code class="file docutils literal notranslate"><span class="pre">_headers</span></code> file and upload it to your website. This will add the <code class="docutils literal notranslate"><span class="pre">X-Robots-Tag:</span> <span class="pre">noindex,</span> <span class="pre">nofollow</span></code> header to the endpoints as described in the <a class="reference external" href="https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag">Google Search Central documentation</a>.</p>
<div class="literal-block-wrapper docutils container" id="id4">
<div class="code-block-caption"><span class="caption-text"><code class="file docutils literal notranslate"><span class="pre">_headers</span></code></span></div>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>/cdn-cgi/bm/cv/*
/cdn-cgi/challenge-platform/*
/cdn-cgi/images/trace/*
/cdn-cgi/rum
/cdn-cgi/scripts/*
/cdn-cgi/styles/*
/cdn-fpw/sxg/*
X-Robots-Tag: noindex, nofollow
</pre></div>
</div>
</div>
<p>After uploading the file to your website, the affected URL can be checked with Web Developer Tools in any recent browser. For example, in Firefox you can open the Web Developer Tools by pressing <code class="docutils literal notranslate"><span class="pre">F12</span></code> and select the Network tab. After that you can reload the page and check the headers for the affected URL.</p>
</section>
</section>
Cloudflare Pages is a great way to host your static website. It’s fast, easy, and free to start with. While the egress traffic is free, you still pay in other forms as some bots will crawl your website. This is where robots.txt comes in handy. You can use it to block bots from crawling your website without having to configure anything with in Cloudflare.The file robots.txt is used to tell bots where to find your sitemap. This is useful for search engines to find your sitemap. The sitemap is used to tell search engines what pages are available on your website. Lets start with a simple example to tell bots where to find your sitemap.2023-07-02T00:00:00+00:00https://dailystuff.nl/blog/2023/minify-websites.htmlHow to minify websites with GitHub Actions2023-04-12T00:00:00+00:00Hans Spaans<section id="how-to-minify-websites-with-github-actions">
<p>Websites are usually made up of HTML, CSS, and Javascript. These files are usually not minified, which means that they are not compressed. This means that the files are not optimized for the web. This can cause the website to load slower than it should. This is especially true for mobile users. This is where minification comes in. Minification is the process of removing unnecessary characters from the files. This can include removing comments, removing whitespace, and removing unnecessary characters. This can make the website load faster and more efficiently.</p>
<p>Multiple ways to minify these files do exist. One way is to use a <a class="reference external" href="https://www.minifier.org/">minifier online</a>. Another way is to use a minifier locally on your computer, but both of these methods are not ideal. A more efficient way is to use a GitHub Action. This is a way to automate tasks on GitHub. This can be used to minify these files before creating an artifact and deploying this artifact to the web.</p>
<p>The application <a class="reference external" href="https://github.com/coderaiser/minify">minify</a> is packaged in <a class="reference external" href="https://github.com/marketplace/actions/minify-action">GitHub Action Minify</a> and can be directly used in a workflow and will reduce the size of all the HTML, CSS, and Javascript files. This can be done by adding lines 21 and 22 like in the workflow example below.</p>
<div class="literal-block-wrapper docutils container" id="id1">
<div class="code-block-caption"><span class="caption-text">Example workflow file <code class="file docutils literal notranslate"><span class="pre">.github/workflows/ci.yml</span></code></span></div>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="linenos"> 1</span><span class="nn">---</span>
<span class="linenos"> 2</span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">CI</span>
<span class="linenos"> 3</span>
<span class="linenos"> 4</span><span class="nt">on</span><span class="p">:</span>
<span class="linenos"> 5</span><span class="nt">push</span><span class="p">:</span>
<span class="linenos"> 6</span><span class="w"> </span><span class="nt">branches</span><span class="p">:</span>
<span class="linenos"> 7</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">master</span>
<span class="linenos"> 8</span><span class="nt">pull_request</span><span class="p">:</span>
<span class="linenos"> 9</span><span class="w"> </span><span class="nt">branches</span><span class="p">:</span>
<span class="linenos">10</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">master</span>
<span class="linenos">11</span>
<span class="linenos">12</span><span class="nt">jobs</span><span class="p">:</span>
<span class="linenos">13</span><span class="w"> </span><span class="nt">build</span><span class="p">:</span>
<span class="linenos">14</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Building Artifacts</span>
<span class="linenos">15</span><span class="w"> </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ubuntu-latest</span>
<span class="linenos">16</span>
<span class="linenos">17</span><span class="w"> </span><span class="nt">steps</span><span class="p">:</span>
<span class="linenos">18</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Checkout Code</span>
<span class="linenos">19</span><span class="w"> </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">actions/checkout@v3</span>
<span class="linenos">20</span>
<span class="hll"><span class="linenos">21</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Minify Action</span>
</span><span class="hll"><span class="linenos">22</span><span class="w"> </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">anthonyftwang/minify-action@v1.0.1</span>
</span><span class="linenos">23</span>
<span class="linenos">24</span><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Upload artifact</span>
<span class="linenos">25</span><span class="w"> </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">actions/upload-artifact@v3</span>
<span class="linenos">26</span><span class="w"> </span><span class="nt">with</span><span class="p">:</span>
<span class="linenos">27</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">html-site</span>
<span class="linenos">28</span><span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">public</span>
<span class="linenos">29</span><span class="w"> </span><span class="nt">retention-days</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1</span>
</pre></div>
</div>
</div>
<p>The example above shows how simple it is to this GitHub Action. The first step is to checkout the code. After this you can run the minify action. This will minify all the HTML, CSS, and Javascript files. The last step is to upload the artifact. This will upload the minified files to the artifact. This can then be used to deploy the website.</p>
</section>
Websites are usually made up of HTML, CSS, and Javascript. These files are usually not minified, which means that they are not compressed. This means that the files are not optimized for the web. This can cause the website to load slower than it should. This is especially true for mobile users. This is where minification comes in. Minification is the process of removing unnecessary characters from the files. This can include removing comments, removing whitespace, and removing unnecessary characters. This can make the website load faster and more efficiently.Multiple ways to minify these files do exist. One way is to use a minifier online. Another way is to use a minifier locally on your computer, but both of these methods are not ideal. A more efficient way is to use a GitHub Action. This is a way to automate tasks on GitHub. This can be used to minify these files before creating an artifact and deploying this artifact to the web.2023-04-12T00:00:00+00:00https://dailystuff.nl/blog/2023/optimize-png-images.htmlHow to optimize PNG images2023-03-26T00:00:00+00:00Hans Spaans<section id="how-to-optimize-png-images">
<p><a class="reference external" href="https://en.wikipedia.org/wiki/PNG">PNG</a> is a lossless image format that supports transparency. It is a good alternative to <a class="reference external" href="https://en.wikipedia.org/wiki/JPEG">JPEG</a> for images with a lot of detail and/or transparency. However, PNG images can be quite large. This post shows how to optimize PNG images instead of converting images to <a class="reference internal" href="../2023/convert-images-to-webp.html#how-to-convert-images-to-webp"><span class="std std-ref">WebP</span></a> as not everyone is ready to publish WebP images yet.</p>
<section id="installing-optipng">
<h2>Installing optipng</h2>
<p>With <a class="reference external" href="https://optipng.sourceforge.net/">optipng</a> you can optimize PNG images. It is available in the repositories of most Linux distributions as shown below for Ubuntu and Debian.</p>
<div class="literal-block-wrapper docutils container" id="id1">
<div class="code-block-caption"><span class="caption-text">Install <code class="docutils literal notranslate"><span class="pre">optipng</span></code> on Ubuntu or Debian</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>optipng
</pre></div>
</div>
</div>
<p>On Fedora and CentOS you can install <code class="docutils literal notranslate"><span class="pre">optipng</span></code> with the following command.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text">Install <code class="docutils literal notranslate"><span class="pre">optipng</span></code> on Fedora or CentOS</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>sudo<span class="w"> </span>dnf<span class="w"> </span>install<span class="w"> </span>optipng
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>An alternative way is to install <a class="reference external" href="https://www.npmjs.com/package/optipng-bin">optipng-bin with npm</a> as it the executable is part of the package/module.</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>npm<span class="w"> </span>install<span class="w"> </span>optipng
</pre></div>
</div>
</div>
</section>
<section id="optimize-png-images">
<h2>Optimize PNG images</h2>
<p>If we take the images same images from a previous post <a class="reference internal" href="../2023/convert-images-to-webp.html#how-to-convert-images-to-webp"><span class="std std-ref">How to Convert Images to WebP</span></a> we can optimize them with <code class="docutils literal notranslate"><span class="pre">optipng</span></code> as shown below and see how much space we can save.</p>
<div class="literal-block-wrapper docutils container" id="id3">
<div class="code-block-caption"><span class="caption-text">Before optimizing images with <code class="docutils literal notranslate"><span class="pre">optipng</span></code></span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>ls<span class="w"> </span>-l<span class="w"> </span>add-issues-to-projects-*
<span class="go">-rw-r--r--. 1 user01 user01 40653 Mar 22 10:48 add-issues-to-projects-1.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 14164 Mar 22 10:49 add-issues-to-projects-1.webp</span>
<span class="go">-rw-r--r--. 1 user01 user01 94538 Mar 22 10:48 add-issues-to-projects-2.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 34864 Mar 22 10:49 add-issues-to-projects-2.webp</span>
</pre></div>
</div>
</div>
<p>The command <code class="docutils literal notranslate"><span class="pre">optipng</span></code> is used to optimize PNG images. It takes the name of the PNG image as an argument. The <code class="docutils literal notranslate"><span class="pre">-o</span></code> option can be used to set the optimization level. The default is <code class="docutils literal notranslate"><span class="pre">2</span></code>. The higher the optimization level the more time it takes to optimize the image. The same goes for additional filter types, but lets start with the default options.</p>
<div class="literal-block-wrapper docutils container" id="id4">
<div class="code-block-caption"><span class="caption-text">Optimize PNG images with <code class="docutils literal notranslate"><span class="pre">optipng</span></code></span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>optipng<span class="w"> </span>add-issues-to-projects-*.png
<span class="go">** Processing: add-issues-to-projects-1.png</span>
<span class="go">983x305 pixels, 4x8 bits/pixel, RGB+alpha</span>
<span class="go">Reducing image to 3x8 bits/pixel, RGB</span>
<span class="go">Input IDAT size = 40438 bytes</span>
<span class="go">Input file size = 40653 bytes</span>
<span class="go">Trying:</span>
<span class="go"> zc = 9 zm = 8 zs = 0 f = 0 IDAT size = 27158</span>
<span class="go">Selecting parameters:</span>
<span class="go"> zc = 9 zm = 8 zs = 0 f = 0 IDAT size = 27158</span>
<span class="go">Output IDAT size = 27158 bytes (13280 bytes decrease)</span>
<span class="go">Output file size = 27324 bytes (13329 bytes = 32.79% decrease)</span>
<span class="go">* Processing: add-issues-to-projects-2.png</span>
<span class="go">983x735 pixels, 4x8 bits/pixel, RGB+alpha</span>
<span class="go">Reducing image to 3x8 bits/pixel, RGB</span>
<span class="go">Input IDAT size = 94239 bytes</span>
<span class="go">Input file size = 94538 bytes</span>
<span class="go">Trying:</span>
<span class="go"> zc = 9 zm = 8 zs = 0 f = 0 IDAT size = 64324</span>
<span class="go">Selecting parameters:</span>
<span class="go"> zc = 9 zm = 8 zs = 0 f = 0 IDAT size = 64324</span>
<span class="go">Output IDAT size = 64324 bytes (29915 bytes decrease)</span>
<span class="go">Output file size = 64490 bytes (30048 bytes = 31.78% decrease)</span>
</pre></div>
</div>
</div>
<p>After optimization we can see that the PNG images are smaller than before and the WebP images are still smaller, but the 30% decrease in size is still significant.</p>
<div class="literal-block-wrapper docutils container" id="id5">
<div class="code-block-caption"><span class="caption-text">Output after optimizing images with <code class="docutils literal notranslate"><span class="pre">optipng</span></code></span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>ls<span class="w"> </span>-l<span class="w"> </span>add-issues-to-projects-*
<span class="go">-rw-r--r--. 1 user01 user01 27324 Mar 26 10:48 add-issues-to-projects-1.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 14164 Mar 22 10:49 add-issues-to-projects-1.webp</span>
<span class="go">-rw-r--r--. 1 user01 user01 64490 Mar 26 10:48 add-issues-to-projects-2.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 34864 Mar 22 10:49 add-issues-to-projects-2.webp</span>
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The command <code class="docutils literal notranslate"><span class="pre">optipng</span></code> has other options like <code class="docutils literal notranslate"><span class="pre">-strip</span> <span class="pre">all</span></code> and <code class="docutils literal notranslate"><span class="pre">-fix</span></code>. The <code class="docutils literal notranslate"><span class="pre">-strip</span> <span class="pre">all</span></code> option can be used to remove all metadata from the image. The <code class="docutils literal notranslate"><span class="pre">-fix</span></code> option can be used to fix some errors in the image. The <code class="docutils literal notranslate"><span class="pre">-quiet</span></code> option can be used to suppress the output.</p>
</div>
</section>
<section id="converting-png-images-to-webp">
<h2>Converting PNG images to WebP</h2>
<p>In previous post <a class="reference internal" href="../2023/convert-images-to-webp.html#how-to-convert-images-to-webp"><span class="std std-ref">How to Convert Images to WebP</span></a> the PNG images were converted to WebP images with the command <code class="docutils literal notranslate"><span class="pre">cwebp</span></code>. But can optimized PNG images be converted to WebP images with <code class="docutils literal notranslate"><span class="pre">cwebp</span></code> and have a differen size. They can still be converted, but the file size doesn’t change.</p>
<div class="literal-block-wrapper docutils container" id="id6">
<div class="code-block-caption"><span class="caption-text">Output after optimizing images with <code class="docutils literal notranslate"><span class="pre">cwebp</span></code></span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>ls<span class="w"> </span>-l<span class="w"> </span>add-issues-to-projects-*
<span class="go">-rw-r--r--. 1 user01 user01 27324 Mar 26 10:48 add-issues-to-projects-1.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 14164 Mar 26 10:49 add-issues-to-projects-1.webp</span>
<span class="go">-rw-r--r--. 1 user01 user01 64490 Mar 26 10:48 add-issues-to-projects-2.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 34864 Mar 26 10:49 add-issues-to-projects-2.webp</span>
</pre></div>
</div>
</div>
<p>The problem here is that optimized images are already compressed and the compression algorithm used by <code class="docutils literal notranslate"><span class="pre">cwebp</span></code> is not able to compress the image further as there is no more data to compress. The only way to reduce the file size is to use a different compression algorithm, but there is a limit to how much the file size can be reduced.</p>
</section>
<section id="run-optipng-with-github-actions">
<h2>Run optipng with GitHub Actions</h2>
<p>Using the command <code class="docutils literal notranslate"><span class="pre">optipng</span></code> to optimize PNG images is easy, but it is not very convenient to run it on a local machine as it doesn’t garantee that the images are optimized. It is better to run it on a CI server like GitHub Actions. The following example shows how to run <code class="docutils literal notranslate"><span class="pre">optipng</span></code> on GitHub Actions before pushing it to a <a class="reference external" href="https://en.wikipedia.org/wiki/Content_delivery_network">content delivery network (CDN)</a>.</p>
<div class="literal-block-wrapper docutils container" id="id7">
<div class="code-block-caption"><span class="caption-text">Add steps to run <code class="docutils literal notranslate"><span class="pre">optipng</span></code> to <code class="file docutils literal notranslate"><span class="pre">.github/workflows/ci.yml</span></code></span></div>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nn">---</span>
<span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">CI</span>
<span class="nt">on</span><span class="p">:</span>
<span class="w"> </span><span class="nt">push</span><span class="p">:</span>
<span class="w"> </span><span class="nt">branches</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">master</span>
<span class="w"> </span><span class="nt">pull_request</span><span class="p">:</span>
<span class="w"> </span><span class="nt">branches</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">master</span>
<span class="nt">jobs</span><span class="p">:</span>
<span class="w"> </span><span class="nt">optipng</span><span class="p">:</span>
<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Optimize and push PNG images</span>
<span class="w"> </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ubuntu-latest</span>
<span class="w"> </span><span class="nt">steps</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Checkout code</span>
<span class="w"> </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">actions/checkout@v2</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Optimize PNG images</span>
<span class="w"> </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">actions/setup-node@v3</span>
<span class="w"> </span><span class="nt">with</span><span class="p">:</span>
<span class="w"> </span><span class="nt">node-version</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">latest</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Optimize PNG images</span>
<span class="w"> </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">|</span>
<span class="w"> </span><span class="no">npm install optipng-bin</span>
<span class="w"> </span><span class="no">npx optipng `git ls-files *\.png`</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Pushing assets to CDN</span>
<span class="w"> </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">|</span>
<span class="w"> </span><span class="no">echo "Do something with the optimized images"</span>
</pre></div>
</div>
</div>
</section>
</section>
PNG is a lossless image format that supports transparency. It is a good alternative to JPEG for images with a lot of detail and/or transparency. However, PNG images can be quite large. This post shows how to optimize PNG images instead of converting images to WebP as not everyone is ready to publish WebP images yet.With optipng you can optimize PNG images. It is available in the repositories of most Linux distributions as shown below for Ubuntu and Debian.2023-03-26T00:00:00+00:00https://dailystuff.nl/blog/2023/convert-images-to-webp.htmlHow to Convert Images to WebP2023-03-25T00:00:00+00:00Hans Spaans<section id="how-to-convert-images-to-webp">
<p>Websites have been using <a class="reference external" href="https://en.wikipedia.org/wiki/PNG">PNG</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/JPEG">JPEG</a> images for a long time. PNG images are lossless and JPEG images are lossy. PNG images are larger in size than JPEG images. WebP is a new image format that is developed by Google. WebP images are smaller in size than PNG and JPEG images and can be both lossy and lossless. <a class="reference external" href="https://en.wikipedia.org/wiki/WebP">WebP</a> images are supported by all modern browsers nowadays and making it ideal see how you can quickly convert images to WebP and make web sites faster and more efficient to load.</p>
<section id="installing-webp-libwebp-tools">
<h2>Installing WebP libwebp-tools</h2>
<p>The first step is to install the <code class="docutils literal notranslate"><span class="pre">webp</span></code> or <code class="docutils literal notranslate"><span class="pre">libwebp-tools</span></code> package. The package includes the command line tools to convert images to WebP and to view WebP images. The package is available in the default repositories of Ubuntu, Debian, Fedora, and CentOS. On Ubuntu and Debian, the package is called <code class="docutils literal notranslate"><span class="pre">webp</span></code>.</p>
<div class="literal-block-wrapper docutils container" id="id1">
<div class="code-block-caption"><span class="caption-text">Install WebP on Ubuntu or Debian</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>sudo<span class="w"> </span>apt<span class="w"> </span>install<span class="w"> </span>webp
</pre></div>
</div>
</div>
<p>On Fedora and CentOS, the package is called <code class="docutils literal notranslate"><span class="pre">libwebp-tools</span></code> instead of <code class="docutils literal notranslate"><span class="pre">webp</span></code> package.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text">Install WebP on Fedora or CentOS</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>sudo<span class="w"> </span>dnf<span class="w"> </span>install<span class="w"> </span>libwebp-tools
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>For other Linux distributions, you can download the source code from the official website and compile it. See the <a class="reference external" href="https://developers.google.com/speed/webp/download">official documentation</a> for more details.</p>
</div>
<p>After installing the <code class="docutils literal notranslate"><span class="pre">webp</span></code> or <code class="docutils literal notranslate"><span class="pre">libwebp-tools</span></code> package, you will have the following command line tools available:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">anim_diff</span></code> - tool to display the difference between animation images.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">anim_dump</span></code> - tool to dump the difference between animation images.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">cwebp</span></code> - webp encoder tool.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">dwebp</span></code> - webp decoder tool.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">gif2webp</span></code> - tool for converting GIF images to webp.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">img2webp</span></code> - tools for converting a sequence of images into an animated webp file.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">vwebp</span></code> - webp file viewer.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">webpinfo</span></code> - used to view info about a webp image file.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">webpmux</span></code> - webp muxing tool.</p></li>
</ul>
</section>
<section id="converting-images-to-webp">
<h2>Converting Images to WebP</h2>
<p>The <code class="docutils literal notranslate"><span class="pre">cwebp</span></code> command line tool is used to convert images to WebP. By default the <code class="docutils literal notranslate"><span class="pre">cwebp</span></code> command line tool converts the images to lossy WebP images at a compression quality of 75 and should be sufficient for most of the images.</p>
<div class="literal-block-wrapper docutils container" id="id3">
<div class="code-block-caption"><span class="caption-text">Convert PNG to WebP</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>cwebp<span class="w"> </span>example.png<span class="w"> </span>-o<span class="w"> </span>example.webp
</pre></div>
</div>
</div>
<p>The quality of lossy compression can be adjusted with the <code class="docutils literal notranslate"><span class="pre">-q</span></code> option. The quality of the WebP images ranges from 0 to 100. In th example below we are converting the PNG image to WebP with a quality of 60.</p>
<div class="literal-block-wrapper docutils container" id="id4">
<div class="code-block-caption"><span class="caption-text">Convert PNG to WebP with reduced quality</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>cwebp<span class="w"> </span>-q<span class="w"> </span><span class="m">60</span><span class="w"> </span>example.png<span class="w"> </span>-o<span class="w"> </span>example.webp
</pre></div>
</div>
</div>
<p>Beside lossy compression, you can also use lossless compression to convert the images to WebP. The lossless compression is slower than the lossy compression. The lossless compression is useful when you want to preserve the quality of the image like when you are converting the images to WebP for archiving purposes.</p>
<div class="literal-block-wrapper docutils container" id="id5">
<div class="code-block-caption"><span class="caption-text">Convert PNG to WebP with lossless compression</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>cwebp<span class="w"> </span>-lossless<span class="w"> </span>example.png<span class="w"> </span>-o<span class="w"> </span>example.webp
</pre></div>
</div>
</div>
</section>
<section id="view-webp-images">
<h2>View WebP Images</h2>
<p>WebP images can be viewed using the <code class="docutils literal notranslate"><span class="pre">vwebp</span></code> command line tool that is included with the libwebp-tools package. The other option to view the WebP images is to use a Chromium-based browser such as <a class="reference external" href="https://www.google.com/chrome/">Google Chrome</a> or <a class="reference external" href="https://www.microsoft.com/nl-nl/edge">Microsoft Edge</a>.</p>
<div class="literal-block-wrapper docutils container" id="id6">
<div class="code-block-caption"><span class="caption-text">View WebP Image</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>vwebp<span class="w"> </span>example.webp
</pre></div>
</div>
</div>
<p>Beside viewing the WebP images, you can also view the metadata of the WebP images using the <code class="docutils literal notranslate"><span class="pre">webpinfo</span></code> command line tool that is included with the libwebp-tools package.</p>
<div class="literal-block-wrapper docutils container" id="id7">
<div class="code-block-caption"><span class="caption-text">View the metadata of a WebP image</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>webpinfo<span class="w"> </span>example.webp
<span class="go">File: example.webp</span>
<span class="go">RIFF HEADER:</span>
<span class="go"> File size: 14164</span>
<span class="go">Chunk VP8 at offset 12, length 14152</span>
<span class="go"> Width: 983</span>
<span class="go"> Height: 305</span>
<span class="go"> Alpha: 0</span>
<span class="go"> Animation: 0</span>
<span class="go"> Format: Lossy (1)</span>
<span class="go">No error detected.</span>
</pre></div>
</div>
</div>
</section>
<section id="the-reduction-in-size">
<h2>The reduction in size</h2>
<p>The size reduction of the WebP images depends on the type of the image. A quick test with images of <a class="reference internal" href="../2023/add-issues-to-projects.html#add-issues-to-projects-on-github"><span class="std std-ref">another blog post</span></a> shows that the size of the WebP images is reduced by 50% to 70% compared to the PNG images.</p>
<div class="literal-block-wrapper docutils container" id="id8">
<div class="code-block-caption"><span class="caption-text">Size of converted PNG to WebP images with reduced quality</span></div>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="gp">$ </span>ls<span class="w"> </span>-l<span class="w"> </span>add-issues-to-projects-*
<span class="go">-rw-r--r--. 1 user01 user01 40653 Mar 22 10:48 add-issues-to-projects-1.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 14164 Mar 22 10:49 add-issues-to-projects-1.webp</span>
<span class="go">-rw-r--r--. 1 user01 user01 94538 Mar 22 10:48 add-issues-to-projects-2.png</span>
<span class="go">-rw-r--r--. 1 user01 user01 34864 Mar 22 10:49 add-issues-to-projects-2.webp</span>
</pre></div>
</div>
</div>
</section>
</section>
Websites have been using PNG and JPEG images for a long time. PNG images are lossless and JPEG images are lossy. PNG images are larger in size than JPEG images. WebP is a new image format that is developed by Google. WebP images are smaller in size than PNG and JPEG images and can be both lossy and lossless. WebP images are supported by all modern browsers nowadays and making it ideal see how you can quickly convert images to WebP and make web sites faster and more efficient to load.The first step is to install the webp or libwebp-tools package. The package includes the command line tools to convert images to WebP and to view WebP images. The package is available in the default repositories of Ubuntu, Debian, Fedora, and CentOS. On Ubuntu and Debian, the package is called webp.2023-03-25T00:00:00+00:00https://dailystuff.nl/blog/2023/implementing-csp.htmlImplementing a Content Security Policy2023-03-21T00:00:00+00:00Hans Spaans<section id="implementing-a-content-security-policy">
<section id="introduction-to-csp">
<h2>Introduction to CSP</h2>
<p>Implementing a <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy">Content Security Policy (CSP)</a> for a website can be a daunting and difficult task as it can break your website when done incorrectly. But a <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy">CSP</a> can help to guard against cross-site script attacks and data injection attacks on a website as it defines which resources are all allowed to be loaded or executed. This also reduces the risk of including unauthorized third-party content to be included or posted to another site.</p>
</section>
<section id="using-csp">
<h2>Using CSP</h2>
<p>Configuring CSP can be done by adding a HTTP response header or by adding a META-tags to the requested HTML content. The latter depends on the CMS to be executed correctly for all HTML content requested. Most sites therefore implement the HTTP response header method as all the common web servers and content delivery networks support this.</p>
<p>The basic syntax for a HTTP response header looks like the example below where the header is named <code class="docutils literal notranslate"><span class="pre">Content-Security-Policy</span></code> and can have one or more policy directive behind it.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text">Systax for the CSP header in enforcing mode</span></div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Content</span><span class="o">-</span><span class="n">Security</span><span class="o">-</span><span class="n">Policy</span><span class="p">:</span> <span class="o"><</span><span class="n">policy</span><span class="o">-</span><span class="n">directive</span><span class="o">></span><span class="p">;</span> <span class="o"><</span><span class="n">policy</span><span class="o">-</span><span class="n">directive</span><span class="o">></span>
</pre></div>
</div>
</div>
<p>The example above directly enforces the policy set in the <code class="docutils literal notranslate"><span class="pre">Content-Security-Policy</span></code> header, but this may break websites when the policy is incomplete. With the header <code class="docutils literal notranslate"><span class="pre">Content-Security-Policy-Report-Only</span></code> the set policy is still active, but only reports violations in the web browsers development tools or sends them to collection service as described in <a class="reference internal" href="../2023/implementing-csp.html#collecting-csp-reports"><span class="std std-ref">Collecting CSP reports</span></a>.</p>
<div class="literal-block-wrapper docutils container" id="id3">
<div class="code-block-caption"><span class="caption-text">Syntax for the CSP header in reporting-only mode</span></div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Content</span><span class="o">-</span><span class="n">Security</span><span class="o">-</span><span class="n">Policy</span><span class="o">-</span><span class="n">Report</span><span class="o">-</span><span class="n">Only</span><span class="p">:</span> <span class="o"><</span><span class="n">policy</span><span class="o">-</span><span class="n">directive</span><span class="o">></span><span class="p">;</span> <span class="o"><</span><span class="n">policy</span><span class="o">-</span><span class="n">directive</span><span class="o">></span>
</pre></div>
</div>
</div>
<p>Another possible option is to include the header as a META-tag in the header of a HTML document. This way isn’t widely used as content management systems has to support this option to generate the correct policy for every HTML document requested.</p>
<div class="literal-block-wrapper docutils container" id="id4">
<div class="code-block-caption"><span class="caption-text">Syntax for the CSP HTML META-tag in enforcing mode</span></div>
<div class="highlight-html notranslate"><div class="highlight"><pre><span></span><span class="p"><</span><span class="nt">meta</span>
<span class="na">http-equiv</span><span class="o">=</span><span class="s">"Content-Security-Policy"</span>
<span class="na">content</span><span class="o">=</span><span class="s">"default-src 'self'; img-src https://*; child-src 'none';"</span> <span class="p">/></span>
</pre></div>
</div>
</div>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Setting up a <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy">Content Security Policy (CSP)</a> and enabling it for a website takes time. Always start with the <code class="docutils literal notranslate"><span class="pre">Content-Security-Policy-Report-Only</span></code> header until no violations are reported anymore or can be explained and have no impact on the functionality of the website.</p>
</div>
</section>
<section id="collecting-csp-reports">
<h2>Collecting CSP reports</h2>
<p>Starting with CSP starts by setting up the directive <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/report-uri">report-uri</a> to collect all violations detected by web browsers. The service called <a class="reference external" href="https://report-uri.com/">Report URI</a> is a good start for most to start with and has a Free-tier with 10.000 free reports per month which should be enough for most small websites.</p>
<p>After generating the URL with <a class="reference external" href="https://report-uri.com/">Report URI</a> a minimal content security policy can be added to the <code class="file docutils literal notranslate"><span class="pre">_headers</span></code> file so Cloudflare Pages can add the header for all content served via Cloudflare Pages.</p>
<div class="literal-block-wrapper docutils container" id="id5">
<div class="code-block-caption"><span class="caption-text">Adding a CSP policy to Cloudflare Pages via the <code class="file docutils literal notranslate"><span class="pre">_headers</span></code> file</span></div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/*</span>
<span class="n">Content</span><span class="o">-</span><span class="n">Security</span><span class="o">-</span><span class="n">Policy</span><span class="o">-</span><span class="n">Report</span><span class="o">-</span><span class="n">Only</span><span class="p">:</span> <span class="n">default</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span><span class="p">;</span> <span class="n">report</span><span class="o">-</span><span class="n">uri</span> <span class="n">https</span><span class="p">:</span><span class="o">//<</span><span class="nb">id</span><span class="o">>.</span><span class="n">report</span><span class="o">-</span><span class="n">uri</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">r</span><span class="o">/</span><span class="n">d</span><span class="o">/</span><span class="n">csp</span><span class="o">/</span><span class="n">reportOnly</span>
</pre></div>
</div>
</div>
<p>Looking at the example more in depth we see that line 1 is for Cloudflare Pages to select for which URLs it needs to serve which headers and here all URLs are being selected. Line 2 shows that directive <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/report-uri">report-uri</a> is set to the reporting URL we got from <a class="reference external" href="https://report-uri.com/">Report URI</a> which takes care of the reporting. The line also shows the HTTP header <code class="docutils literal notranslate"><span class="pre">Content-Security-Policy-Report-Only</span></code> for the browser to only report violations for the policy that is defined behind it in the line.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>All examples are written for Cloudflare Pages, but the example can be adjusted as shown below for both Apache HTTPd and Nginx.</p>
<div class="literal-block-wrapper docutils container" id="id6">
<div class="code-block-caption"><span class="caption-text">Example configuration for Apache HTTPd</span></div>
<div class="highlight-apache notranslate"><div class="highlight"><pre><span></span><span class="nt"><Location</span><span class="w"> </span><span class="s">"/"</span><span class="nt">></span>
<span class="nb">Header</span><span class="w"> </span>always<span class="w"> </span>set<span class="w"> </span>Content-Security-Policy-Report-Only<span class="w"> </span><span class="s2">"default-src 'self'; report-uri https://<id>.report-uri.com/r/d/csp/reportOnly"</span>
<span class="nt"></Location></span>
</pre></div>
</div>
</div>
<div class="literal-block-wrapper docutils container" id="id7">
<div class="code-block-caption"><span class="caption-text">Example configuration for Nginx</span></div>
<div class="highlight-nginx notranslate"><div class="highlight"><pre><span></span><span class="k">add_header</span><span class="w"> </span><span class="s">Content-Security-Policy</span><span class="w"> </span><span class="s">"default-src</span><span class="w"> </span><span class="s">'self'</span><span class="p">;</span><span class="w"> </span><span class="k">report-uri</span><span class="w"> </span><span class="s">https://<id>.report-uri.com/r/d/csp/reportOnly"</span><span class="w"> </span><span class="s">always</span><span class="p">;</span>
</pre></div>
</div>
</div>
</div>
</section>
<section id="extending-the-csp-policy">
<h2>Extending the CSP policy</h2>
<p>In the previous section a basic policy was defined and all violations for that policy are reported by the browser. Over time <a class="reference external" href="https://report-uri.com/">Report URI</a> will be filled with all violations and those can be added to the policy as they’re allowed, or solved by removing them from the site, or being accepted as valid violations that must be stopped by the browser. One of the most common policies to add is for <a class="reference external" href="https://developers.google.com/tag-platform/security/guides/csp">Google Tag-Manager</a> to allow analytics to be collected. The example below has been extended to allow Google Analytics and Adsense to work correctly.</p>
<div class="literal-block-wrapper docutils container" id="id8">
<div class="code-block-caption"><span class="caption-text">Extending the CSP policy file <code class="file docutils literal notranslate"><span class="pre">_headers</span></code> for Google Analytics and Adsense</span></div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/*</span>
<span class="n">Content</span><span class="o">-</span><span class="n">Security</span><span class="o">-</span><span class="n">Policy</span><span class="o">-</span><span class="n">Report</span><span class="o">-</span><span class="n">Only</span><span class="p">:</span> <span class="n">default</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span><span class="p">;</span> <span class="n">style</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="s1">'unsafe-inline'</span><span class="p">;</span> <span class="n">script</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">script</span><span class="o">-</span><span class="n">src</span><span class="o">-</span><span class="n">elem</span> <span class="s1">'self'</span> <span class="s1">'unsafe-inline'</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">unpkg</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">static</span><span class="o">.</span><span class="n">cloudflareinsights</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">img</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">-</span><span class="n">analytics</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">analytics</span><span class="o">.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">g</span><span class="o">.</span><span class="n">doubleclick</span><span class="o">.</span><span class="n">net</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">nl</span><span class="p">;</span> <span class="n">connect</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">-</span><span class="n">analytics</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">analytics</span><span class="o">.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">g</span><span class="o">.</span><span class="n">doubleclick</span><span class="o">.</span><span class="n">net</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">nl</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">cloudflareinsights</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">report</span><span class="o">-</span><span class="n">uri</span> <span class="n">https</span><span class="p">:</span><span class="o">//<</span><span class="nb">id</span><span class="o">>.</span><span class="n">report</span><span class="o">-</span><span class="n">uri</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">r</span><span class="o">/</span><span class="n">d</span><span class="o">/</span><span class="n">csp</span><span class="o">/</span><span class="n">reportOnly</span>
</pre></div>
</div>
</div>
</section>
<section id="enforcing-csp">
<h2>Enforcing CSP</h2>
<p>Enforcing a Content-Security-Policy isn’t required for sites to function correctly and for static websites the usefulness is questionable as it must be maintained. A lot of website currently only have their Content-Security-Policy in Report-Only mode. The example below is to enable the Content-Security-Policy and violations are submitted to a different URL to mark them as a policy violation.</p>
<div class="literal-block-wrapper docutils container" id="id9">
<div class="code-block-caption"><span class="caption-text">Enforcing a CSP policy via the <code class="file docutils literal notranslate"><span class="pre">_headers</span></code> file</span></div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/*</span>
<span class="n">Content</span><span class="o">-</span><span class="n">Security</span><span class="o">-</span><span class="n">Policy</span><span class="p">:</span> <span class="n">default</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span><span class="p">;</span> <span class="n">style</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="s1">'unsafe-inline'</span><span class="p">;</span> <span class="n">script</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">script</span><span class="o">-</span><span class="n">src</span><span class="o">-</span><span class="n">elem</span> <span class="s1">'self'</span> <span class="s1">'unsafe-inline'</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">unpkg</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">static</span><span class="o">.</span><span class="n">cloudflareinsights</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">img</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">-</span><span class="n">analytics</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">analytics</span><span class="o">.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">g</span><span class="o">.</span><span class="n">doubleclick</span><span class="o">.</span><span class="n">net</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">nl</span><span class="p">;</span> <span class="n">connect</span><span class="o">-</span><span class="n">src</span> <span class="s1">'self'</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">-</span><span class="n">analytics</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">analytics</span><span class="o">.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">googletagmanager</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">g</span><span class="o">.</span><span class="n">doubleclick</span><span class="o">.</span><span class="n">net</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">https</span><span class="p">:</span><span class="o">//*.</span><span class="n">google</span><span class="o">.</span><span class="n">nl</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">cloudflareinsights</span><span class="o">.</span><span class="n">com</span><span class="p">;</span> <span class="n">report</span><span class="o">-</span><span class="n">uri</span> <span class="n">https</span><span class="p">:</span><span class="o">//<</span><span class="nb">id</span><span class="o">>.</span><span class="n">report</span><span class="o">-</span><span class="n">uri</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">r</span><span class="o">/</span><span class="n">d</span><span class="o">/</span><span class="n">csp</span><span class="o">/</span><span class="n">enforce</span>
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Currently no technical requirements are known to enforce a content security policy. There could be a regulatory requirement to do so, or to reduce the risk profile. It is advised to consult local legal advice for this.</p>
</div>
</section>
<section id="future-changes">
<h2>Future changes</h2>
<p>While the directive <code class="docutils literal notranslate"><span class="pre">report-uri</span></code> has been deprecated in favor of directive <code class="docutils literal notranslate"><span class="pre">report-to</span></code>, the new directive hasn’t been widely adopted. The lack of adoption makes it difficult to switch as <a class="reference external" href="https://report-uri.com/">Report URI</a> doesn’t offer support yet, but also browsers don’t support the directive yet. For now it is advised to use the <code class="docutils literal notranslate"><span class="pre">report-uri</span></code> directive until services like <a class="reference external" href="https://report-uri.com/">Report URI</a> offer support.</p>
</section>
</section>
Implementing a Content Security Policy (CSP) for a website can be a daunting and difficult task as it can break your website when done incorrectly. But a CSP can help to guard against cross-site script attacks and data injection attacks on a website as it defines which resources are all allowed to be loaded or executed. This also reduces the risk of including unauthorized third-party content to be included or posted to another site.Configuring CSP can be done by adding a HTTP response header or by adding a META-tags to the requested HTML content. The latter depends on the CMS to be executed correctly for all HTML content requested. Most sites therefore implement the HTTP response header method as all the common web servers and content delivery networks support this.2023-03-21T00:00:00+00:00