# How I automate downloading of application installers using PowerShell

# Background

Working in air-gapped environment means not having the luxury to rely on respective application updater to check for newer version automatically. So I needed to manually download the installers from time to time, which is quite time-consuming and a mundane task.

I usually automate repetitive and mundane work, and I have always wanted to automate this process, but I wasn't able to figure out an effective way to do so because (as far as I know) there is no standardized way of grabbing the download URL from various websites. Let's take a look at some samples.

```json
[
  {
    "application": "7zip.7zip",
    "download_url": "https://www.7-zip.org/a/7z2200-x64.exe"
  },
  {
    "application": "Docker.DockerDesktop",
    "download_url": "https://desktop.docker.com/win/main/amd64/81317/Docker%20Desktop%20Installer.exe"
  },
  {
    "application": "Git.Git",
    "download_url": "https://github.com/git-for-windows/git/releases/download/v2.37.0.windows.1/Git-2.37.0-64-bit.exe"
  },
  {
    "application": "GitExtensionsTeam.GitExtensions",
    "download_url": "https://github.com/gitextensions/gitextensions/releases/download/v3.5.4/GitExtensions-3.5.4.12724-65f01f399.msi"
  },
  {
    "application": "Google.Chrome",
    "download_url": "https://dl.google.com/dl/chrome/install/googlechromestandaloneenterprise64.msi"
  }
]
```

There is no fixed templates / format to the URL, and there isn't a way to query for the download URL except for some like `Cypress` that provides a [json file](https://download.cypress.io/desktop.json) with the version and URL.

# Ideas

This thing has always been at the back of my mind for a number of years, and I imagine maintaining the URL myself like such

```json
{
    "application": "Google.Chrome",
    "version": "103.0.5060.66",
    "download_url": "https://dl.google.com/dl/chrome/install/googlechromestandaloneenterprise64.msi"
}
```

But I would need to manually update this as well, and doing so, would take even longer time than having to visit each website to download the installers myself. Hence, I didn't pursue the option since it didn't quite value-add for me.

Some other idea I had was to write scripts using `cypress` to navigate through the page to download various applications, but this will break easily if the website gets updated / changed. So the maintenance cost is high as well.

# Winget

That is, until I came across [winget](https://docs.microsoft.com/en-us/windows/package-manager/winget/). `winget` is a package manager for windows from Microsoft, it allows user to easily manage applications via `winget` just like you would on Linux using `apt-get/yum`.

Listing all the installed applications on my machine would look like

```
╰─ winget list --source winget
Name                                        Id                                           Version             Available
-----------------------------------------------------------------------------------------------------------------------
StarUML                                     MKLabs.StarUML                               4.1.6
P3X Redis UI                                PatrikLaszlo.P3XRedisUI                      2022.4.116          2022.4.126
Visual Studio Build Tools 2017              2fd33d5a                                     15.9.28307.1033
```

An example of a manifest for `StarUML` would look like

```
╰─ winget show "MKLabs.StarUML"
Found StarUML [MKLabs.StarUML]
Version: 4.1.6
Publisher: MKLabs Co.,Ltd.
Publisher Support Url: https://staruml.io/support
Author: MKLabs Co.,Ltd
Moniker: staruml
Description: A sophisticated software modeler for agile and concise modeling.
Homepage: https://staruml.io/
License: Proprietary
License Url: https://staruml.io/eula
Copyright: Copyright (c) 2021 MKLabs Co.,Ltd.
Installer:
  Type: nullsoft
  Download Url: https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe
  SHA256: dd894019785afc4a49c1b18593ed2e0059ed28b6ce3bf85da0415da59a7a3d6d
```

While it is easy to list / install / upgrade the application via `winget-cli`, it is not as easy to parse the content as it is not a native PowerShell object, nor can it be exported to a `json` format. There are [issue](https://github.com/microsoft/winget-cli/issues/221) and [discussion](https://github.com/microsoft/winget-cli/discussions/2300) in `GitHub` tracking this closely, and I hope that these will be supported soon.

Nevertheless, I spent some time recently to write a simple PowerShell script to parse it, and it worked quite nicely.

# Solution

I am not well verse in PowerShell script, so while it works, it may not be the best way to write it. I don't really care so much since it does its job as far as I'm concern.

Referring back to the `StarUML` manifest above. The key is to extract out the URL, and then download it.

This is the command to run to extract the URL

```powershell
$url=winget show "MKLabs.StarUML" | Select-String "Download Url: " -NoEmphasis | Out-String | % { $_.Trim() } | % { $_.SubString(14) }
```

Let's take a look at how this command unfolds.

> I will be showing the input and output in a single codebox

- Let's fetch the manifest of the application

```
> winget show "MKLabs.StarUML"

Found StarUML [MKLabs.StarUML]
Version: 4.1.6
Publisher: MKLabs Co.,Ltd.
Publisher Support Url: https://staruml.io/support
Author: MKLabs Co.,Ltd
Moniker: staruml
Description: A sophisticated software modeler for agile and concise modeling.
Homepage: https://staruml.io/
License: Proprietary
License Url: https://staruml.io/eula
Copyright: Copyright (c) 2021 MKLabs Co.,Ltd.
Installer:
  Type: nullsoft
  Download Url: https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe
  SHA256: dd894019785afc4a49c1b18593ed2e0059ed28b6ce3bf85da0415da59a7a3d6d
```

- Use `Select-String` to search for the text we want

```
> winget show "MKLabs.StarUML" | Select-String "Download Url: " -NoEmphasis

  Download Url: https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe

```

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1656866132082/EbIP_HUWU.png align="left")

- Convert output into `String`, otherwise, we can't apply `String` operation later on

```
> winget show "MKLabs.StarUML" | Select-String "Download Url: " -NoEmphasis | Out-String


  Download Url: https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe

```

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1656863842344/UgAbugPCz.png align="left")

The output look exactly the same as before, but in `String` format. If we don't, when we apply `String` operation, it will throw the following error

```
InvalidOperation: Method invocation failed because [Microsoft.PowerShell.Commands.MatchInfo] does not contain a method named 'Trim'.
```

- Apply `Trim` operation to remove all whitespaces

```
> winget show "MKLabs.StarUML" | Select-String "Download Url: " -NoEmphasis | Out-String | % { $_.Trim() } 

Download Url: https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe
```

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1656863854312/8C_qSQR0l.png align="left")

> % is a shorthand for `ForEach-Object`

- Apply `SubString` operation to grab only the actual URL

```
> winget show "MKLabs.StarUML" | Select-String "Download Url: " -NoEmphasis | Out-String | % { $_.Trim() } | % { $_.SubString(14) }

https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe
```

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1656864191580/rIfzCOwch.png align="left")

And there we have it!

- To download, we can use `Invoke-WebRequest`

```powershell
Invoke-WebRequest -URI https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe -OutFile ./staruml.exe
```

But it is very slow in download, and we have to specify explicit filename. So let's use `curl` instead

```curl
$default_download_dir="./_winget_applications"
$url=https://staruml.io/download/releases-v4/StarUML%20Setup%204.1.6.exe

# -L: follow redirect
# -O -J: we want to retain the remote filename instead of constructing our own
# see https://daniel.haxx.se/blog/2020/09/10/store-the-curl-output-over-there
# --create-dirs: if not exist
# --silent: do not show progress
curl -L $url -O -J --output-dir $default_download_dir --create-dirs --silent
```

So to recap, what we have done is to

- grab the URL from the manifest
- use curl to download the file

And what we have to do is to repeat the process for a list of applications. That's easy, just put it in a loop, and we're done.

For the complete script, please refer to my [GitHub repository](#source-code).

# Conclusion

We looked at the step-by-step process of how I automate the mundane task of downloading application installers using `PowerShell` script with the help of `winget-cli`. However, this is not a foolproof solution as there are still a number of applications that have yet to onboard to `winget`, and so I will not be able to use this method to download those installers automatically.

I also hope to extend this automation to IDE plugins as well, which I have done so in the past for [vscode](https://github.com/bwgjoseph/vscode-extension-downloader) but I now realize that there are much easier way to achieve the same goal.

# Source Code

As usual, full source code is available in [GitHub](https://github.com/bwgjoseph/downloader).