One question that we see once in a while in forums is how to download a file from a HTTPS site.
Luckily, .Net has everything we need. No special 3rd party library are required.
The reason I had to execute code like this was to download the status of some backups from CrashPlan Pro – a useful cloud-backup provider which I have been personally using for quite some time (notice that I don’t get any benefits from them!).
This month’s downloadable code
This month solution contains both VB and C# projects. The solution was created using Visual Studio 2017 but the code should work as well in older version of the.Net Framework.
Figure 1: The demo application in action
The WebRequest namespace
It is not the first time I am using the WebRequest namespace in an article.
I used it in August 2010 to consume RSS feeds.
It was also used in March 2004 to download contents from regular HTTP source.
What this code does?
The code here will be very simple. It is a single method. It will connect to a HTTPS URL with all the arguments that you would usually get from clicking the various options on the web site after being authenticated.
The resulting file, a CSV file in my case, will be saved locally in the temp folder.
The trick here is to pass the credentials to the page. That mechanism can vary depending on how the web site was created.
I was lucky that CrashPlan Pro was accepting the credentials (user id and password) to be passed in the headers of the request object once encoded with 64-bits digits.
Private Sub TransferFile()
Try
ListBox1.Items.Add("Starting the file transfer")
'Your variables need to be filled with your values
Dim strURL As String = "https://www.crashplanpro.com/api/Computer?orgUid=YourORGIDGoesHere&export='csv'&incActivity=true&incBackupUsage=true"
Dim strUID As String = "YourUserIDGoesHere"
Dim strPWD As String = "YourPasswordGoesHere"
'creating a web request object
Dim req As WebRequest = WebRequest.Create(strURL)
'creating the credentials adding them to the header
Dim cre As String = $"{strUID}:{strPWD}"
Dim bytes() As Byte = Encoding.ASCII.GetBytes(cre)
Dim base64 As String = Convert.ToBase64String(bytes)
req.Headers.Add("Authorization", "basic " + base64)
'query the web site for the response to our request
Dim resp As WebResponse = req.GetResponse()
'Create the local file
Dim strDest As String = Path.Combine(Path.GetTempPath(), Date.Now.ToString("yyyyMMdd_HHmmss_") + "Crashplan.csv")
'process the response
Dim arrBuffer(16384) As Byte
Dim intBytesRead As Integer
Using stmLocalFile As FileStream = New FileStream(strDest, FileMode.Create)
With resp.GetResponseStream
Do
intBytesRead = .Read(arrBuffer, 0, 16384)
stmLocalFile.Write(arrBuffer, 0, intBytesRead)
Loop Until intBytesRead = 0
.Close()
'Force all data out of memory and into the file before closing
stmLocalFile.Flush()
stmLocalFile.Close()
End With
End Using
ListBox1.Items.Add("File downloaded")
ListBox1.Items.Add(strDest)
Catch ex As Exception
ListBox1.Items.Add("Something went wrong...")
ListBox1.Items.Add(ex.Message)
End Try
End Sub
private void TransferFile()
{
try
{
ListBox1.Items.Add("Starting the file transfer");
//Your variables need to be filled with your values
string strURL = "https://www.crashplanpro.com/api/Computer?orgUid=YourORGIDGoesHere&export='csv'&incActivity=true&incBackupUsage=true";
string strUID = "YourUserIDGoesHere";
string strPWD = "YourPasswordGoesHere";
//creating a web request object
WebRequest req = WebRequest.Create(strURL);
//creating the credentials adding them to the header
string cre = $"{strUID}:{strPWD}";
byte[] bytes = Encoding.ASCII.GetBytes(cre);
string base64 = Convert.ToBase64String(bytes);
req.Headers.Add("Authorization", "basic " + base64);
//query the web site for the response to our request
WebResponse resp = req.GetResponse();
//Create the local file
string strDest = Path.Combine(Path.GetTempPath(), DateTime.Now.ToString("yyyyMMdd_HHmmss_") + "Crashplan.csv");
//process the response
byte[] arrBuffer = new byte[16385];
int intBytesRead;
using (FileStream stmLocalFile = new FileStream(strDest, FileMode.Create))
{
var respStream = resp.GetResponseStream();
do
{
intBytesRead = respStream.Read(arrBuffer, 0, 16384);
stmLocalFile.Write(arrBuffer, 0, intBytesRead);
} while (intBytesRead != 0);
respStream.Close();
//Force all data out of memory and into the file before closing
stmLocalFile.Flush();
stmLocalFile.Close();
}
ListBox1.Items.Add("File downloaded");
ListBox1.Items.Add(strDest);
}
catch (Exception e)
{
ListBox1.Items.Add("Something went wrong...");
ListBox1.Items.Add(e.Message);
}
}
Conclusion
I needed a way to get the status of my backups sent to CrashPlan Pro. Of course, they are offering emails with those statuses but I wanted to integrate the results in a monitoring dashboard showing the “health” of my system.
The WebRequest namespace has everything I needed to connect to a protected HTTPS site and get the report I needed.