One question that we see once in a while in forums is how to download a file from a HTTPS site.
Luckily, .Net has everything we need. No special 3rd party library are required.
The reason I had to execute code like this was to download the status of some backups from CrashPlan Pro – a useful cloud-backup provider which I have been personally using for quite some time (notice that I don’t get any benefits from them!).
This month’s downloadable code
This month solution contains both VB and C# projects. The solution was created using Visual Studio 2017 but the code should work as well in older version of the.Net Framework.
Figure 1: The demo application in action
The WebRequest namespace
It is not the first time I am using the WebRequest namespace in an article.
I used it in August 2010 to consume RSS feeds.
It was also used in March 2004 to download contents from regular HTTP source.
What this code does?
The code here will be very simple. It is a single method. It will connect to a HTTPS URL with all the arguments that you would usually get from clicking the various options on the web site after being authenticated.
The resulting file, a CSV file in my case, will be saved locally in the temp folder.
The trick here is to pass the credentials to the page. That mechanism can vary depending on how the web site was created.
I was lucky that CrashPlan Pro was accepting the credentials (user id and password) to be passed in the headers of the request object once encoded with 64-bits digits.
Private Sub TransferFile() Try ListBox1.Items.Add("Starting the file transfer") 'Your variables need to be filled with your values Dim strURL As String = "https://www.crashplanpro.com/api/Computer?orgUid=YourORGIDGoesHere&export='csv'&incActivity=true&incBackupUsage=true" Dim strUID As String = "YourUserIDGoesHere" Dim strPWD As String = "YourPasswordGoesHere" 'creating a web request object Dim req As WebRequest = WebRequest.Create(strURL) 'creating the credentials adding them to the header Dim cre As String = $"{strUID}:{strPWD}" Dim bytes() As Byte = Encoding.ASCII.GetBytes(cre) Dim base64 As String = Convert.ToBase64String(bytes) req.Headers.Add("Authorization", "basic " + base64) 'query the web site for the response to our request Dim resp As WebResponse = req.GetResponse() 'Create the local file Dim strDest As String = Path.Combine(Path.GetTempPath(), Date.Now.ToString("yyyyMMdd_HHmmss_") + "Crashplan.csv") 'process the response Dim arrBuffer(16384) As Byte Dim intBytesRead As Integer Using stmLocalFile As FileStream = New FileStream(strDest, FileMode.Create) With resp.GetResponseStream Do intBytesRead = .Read(arrBuffer, 0, 16384) stmLocalFile.Write(arrBuffer, 0, intBytesRead) Loop Until intBytesRead = 0 .Close() 'Force all data out of memory and into the file before closing stmLocalFile.Flush() stmLocalFile.Close() End With End Using ListBox1.Items.Add("File downloaded") ListBox1.Items.Add(strDest) Catch ex As Exception ListBox1.Items.Add("Something went wrong...") ListBox1.Items.Add(ex.Message) End Try End Sub
private void TransferFile() { try { ListBox1.Items.Add("Starting the file transfer"); //Your variables need to be filled with your values string strURL = "https://www.crashplanpro.com/api/Computer?orgUid=YourORGIDGoesHere&export='csv'&incActivity=true&incBackupUsage=true"; string strUID = "YourUserIDGoesHere"; string strPWD = "YourPasswordGoesHere"; //creating a web request object WebRequest req = WebRequest.Create(strURL); //creating the credentials adding them to the header string cre = $"{strUID}:{strPWD}"; byte[] bytes = Encoding.ASCII.GetBytes(cre); string base64 = Convert.ToBase64String(bytes); req.Headers.Add("Authorization", "basic " + base64); //query the web site for the response to our request WebResponse resp = req.GetResponse(); //Create the local file string strDest = Path.Combine(Path.GetTempPath(), DateTime.Now.ToString("yyyyMMdd_HHmmss_") + "Crashplan.csv"); //process the response byte[] arrBuffer = new byte[16385]; int intBytesRead; using (FileStream stmLocalFile = new FileStream(strDest, FileMode.Create)) { var respStream = resp.GetResponseStream(); do { intBytesRead = respStream.Read(arrBuffer, 0, 16384); stmLocalFile.Write(arrBuffer, 0, intBytesRead); } while (intBytesRead != 0); respStream.Close(); //Force all data out of memory and into the file before closing stmLocalFile.Flush(); stmLocalFile.Close(); } ListBox1.Items.Add("File downloaded"); ListBox1.Items.Add(strDest); } catch (Exception e) { ListBox1.Items.Add("Something went wrong..."); ListBox1.Items.Add(e.Message); } }
Conclusion
I needed a way to get the status of my backups sent to CrashPlan Pro. Of course, they are offering emails with those statuses but I wanted to integrate the results in a monitoring dashboard showing the “health” of my system.
The WebRequest namespace has everything I needed to connect to a protected HTTPS site and get the report I needed.