My images still too big. I converted the images in one post and saw good results, but the rest of the posts are filled up with huge images in non-web efficient formats. What to do? Script a solution!

My Images Are Too Big; or, Google Page Speed is Your Friend Part 2

Google Page Speed

Let’s pick up where we left off:

Performance Improvement

Good, but we can do better.

I need to address the images on my site that are not using webp, which is all of them.

To do this by hand would suck, and why would anyone do that when scripting up a solution would be faster, repeatable, and more fun?

Scripting a Solution

I’ve never done a lot of scripting in my career. Up until recently, I’d been strictly a coder (not that scripting isn’t coding), working my lane, on a small piece of something larger that involved infrastructure, security, and the scripts that go hand-in-hand with it all; or, designing pieces that fit together.

That changed when I worked at an agency. Given the people count was low, development was done much in the spirit of the original meaning of DevOps

DevOps is a combination of software developers (dev) and operations (ops). It is defined as a software engineering methodology which aims to integrate the work of software development and software operations teams by facilitating a culture of collaboration and shared responsibility. https://about.gitlab.com/topics/devops/

Devs were expected to know about all areas that were touched by a piece of software, come up with solutions for maintaining and administering them, and they were given the power to implement those solutions. Developers were trusted, and barriers were removed to get things done quickly.

Back to the task at hand: my first thought was to use bash, as I like digging around in Linux. I like Linux in general, despite being a beginner.

Work, however, is a Microsoft shop with a few exceptions; that means Powershell.

I don’t know Powershell.

Time to get to know Powershell.

The Plan

Time to write a set of steps to get me where I want to go. Tech is secondary, the problem is everything.

1. Get all the images in my repository that aren't webp.
2. For each image, convert it to webp.
3. Find all references in the markdown files, and change each reference to the new filename (.webp)

Bob’s your uncle.

Time to Powershell up this bad boy and get my blog performing like a thoroughbred instead of an old nag.

Powershelling Like a Boss

Since I knew nothing, it’s more like Googling like a boss, amiright?

How to get all files/filenames?

Learning Powershell

Going in blind? Have a look at the Learning Powershell guide at Microsoft.

It’s what you want.

Getting Files

How to get files?

Get-ChildItem is your guy.

The Get-ChildItem cmdlet gets the items in one or more specified locations. If the item is a container, it gets the items inside the container, known as child items. You can use the Recurse parameter to get items in all child containers and use the Depth parameter to limit the number of levels to recurse. Get-ChildItem doesn’t display empty directories. When a Get-ChildItem command includes the Depth or Recurse parameters, empty directories aren’t included in the output. Locations are exposed to Get-ChildItem by PowerShell providers. A location can be a file system directory, registry hive, or a certificate store. For more information, see about_Providers.

For Each

How do I loop? How do I for each?

This is how: https://devblogs.microsoft.com/scripting/basics-of-powershell-looping-foreach/

Look pretty straightforward.

Two pieces of the puzzle down.

Looping then would be like this:

// find all assets that are non-webp images
foreach ($asset in $assets) {
  // convert asset to webp
}

Converting Images

Webp, where for art thou?

Here thou art: https://developers.google.com/speed/webp

A command-line tool to convert images, now we’re getting somewhere.

What this tells me is that I need to add a parameter for passing in the path of the webp converter. I could add the path to system path, but better to assume I might not want to or have done that.

Param(
    [Parameter(Mandatory = $true)]
    [string]$webpPath
)

I’ve made the parameter mandatory, because I want the user to always have to tell the script where the webp converter is located.

Changing Text in the Markdown Files

Once the new webp files are in place, I need to find all the references to the old files in the markdown and change their extensions to .webp, otherwise all the links will break.

function Find-And-Replace-In-Files {
    param (
        [string]$textToFind,
        [string]$textToReplace,
        [string[]]$filesToSearch
    )

    Write-Host Finding> $textToFind
    Write-Host Replacing> $textToReplace

    foreach ($fileToSearch in $filesToSearch) {
        $content = get-content $fileToSearch
        $fullTextToFind = "assets/img/posts/" + $textToFind;

        if ($content -match $fullTextToFind) {
            $content | ForEach-Object { $_ -replace $textToFind, $textToReplace } | Set-Content $fileToSearch
        }
    }
}

Putting it All Together

All the pieces should be in place for a messy first draft of a script. Here goes!


Param(
    [Parameter(Mandatory = $true)]
    [string]$webpPath
)

function Find-And-Replace-In-Files {
    param (
        [string]$textToFind,
        [string]$textToReplace,
        [string[]]$filesToSearch
    )

    Write-Host Finding> $textToFind
    Write-Host Replacing> $textToReplace

    foreach ($fileToSearch in $filesToSearch) {
        $content = get-content $fileToSearch
        $fullTextToFind = "assets/img/posts/" + $textToFind;

        if ($content -match $fullTextToFind) {
            $content | ForEach-Object { $_ -replace $textToFind, $textToReplace } | Set-Content $fileToSearch
        }
    }
}

# set path to webp converter
$webpExeFullPath = $webpPath + "\cwebp.exe";

Write-Host "Path to webp converter: " $webpExeFullPath;

$markdownPosts = Get-ChildItem .\_posts -Include ('*.md', '*.markdown') -recurse; 
# get all jpg and png files
$assets = Get-ChildItem .\* -Include ('*.jpg', '*.jpeg', '*.png') -recurse;

foreach ($asset in $assets) {
    # replace the file extension with webp for the new file
    $assetWebpFilenameFullPath = $asset.Directory.ToString() + "\" + $asset.Name.Substring(0, $asset.Name.Length - $asset.Extension.Length) + '.webp';
    $assetWebpFilename = "$($asset.Name.Substring(0, $asset.Name.Length - $asset.Extension.Length)).webp";
    # create the arguments for the command in the format:
    # cwebp [options] -q quality input.png -o output.webp
    # quality defaults to 75
    # image files must be in doublequotes in windows.
    Write-Host `n****************`nProcessing $asset...
    Write-Host New Filename: $assetWebpName;
    $arguments = "`"$($asset)`" -o `"$assetWebpFilenameFullPath`""
    Write-Host $arguments
    Start-Process -Wait $webpExeFullPath -NoNewWindow -PassThru $arguments
    Write-Host Done Processing $asset.
    Find-And-Replace-In-Files -textToFind $asset.Name -textToReplace $assetWebpFilename -filesToSearch $markdownPosts
    Write-Host ****************`n
}

Time to let it rip, and then clean up later!

The End

It worked!

Everything converted! I checked in all the changes, deleted the non-webp images, and pushed it up.

Phew!

It wasn’t pretty, but it was fun, and it got the job done.

There’s lots more to dig into, like accessibility, the gifs being massive, and much more.

Until then, keep improving every day!