STARLIMS + Azure Blob containers through SAS

STARLIMS + Azure Blob containers through SAS

Here is a funny story.

Here I was trying, with the infra team, to access my Azure container through SAS. STARLIMS has a built-in Azure container support, but it relies on a connection string with account information and all. But, like most Azure customers, that is not our reality. We use shared containers, so we need a SAS token… Which is not supported to configure as a STARLIMS connection string.

This means that next step will be any other web service consumption instead of direct containers access. Is it complex? Less than I expected!

Step 1: let’s get a SAS token!

Now, finding the said token is not always obvious, but mine looked something like this:

sv=nnnn-nn-nn&sr=c&si=CompanyaccessPolicy&sig=someuglystring

Hopefully, yours too! In the Azure Container tool, look for the “Shared Access Signature”, it’s the same thing.

Step 2 – integrate the Azure API!

Now, how do we put files there? The connection string and tutorials on STARLIMS will not help… But the web services will! All we need to do is write a UploadToAzureBlob procedure and a DownloadFromAzureBlob procedure (both in SSL) and that will do the trick:

:PROCEDURE UploadToAzureBlob;
:PARAMETERS content, fileName;

:DECLARE    sasToken, storageAccount, containerName, method, sampleContent,
            contentLength, requestUri, oWebService, oClient, oRequest, innerRequest,
            stream, resp, encoding;

encoding := LimsNetConnect("", "System.Text.Encoding",, .T.);


sasToken := "yourSASToken";
storageAccount := "yourAccountName";
containerName := "yourContainerName";
blobName := fileName; /* can be something like folder/subfolder/name.ext ;

method := "PUT";
sampleContent := content;
contentLength := encoding:UTF8:GetByteCount(sampleContent);

requestUri := Replace(Replace(Replace(Replace(
                    "https://{storageAccount}.blob.core.windows.net/{containerName}/{blobName}?{sasToken}",
                        "{storageAccount}", storageAccount),
                        "{containerName}", containerName),
                        "{blobName}", blobName),
                        "{sasToken}", sasToken);

oWebService := WebServices{};
oClient := oWebService:CreateHttpClient();
oRequest := oClient:CreateHttpRequest(requestUri);

oRequest:Method := method;
oRequest:ContentType := "text/plain; charset=UTF-8";
oRequest:ContentLength := contentLength;

innerRequest := DoProc("GetInnerRequest", { oRequest });
innerRequest:Headers:Add("x-ms-blob-type", "BlockBlob");
stream := innerRequest:GetRequestStream();
stream:Write(encoding:UTF8:GetBytes(sampleContent), 0, contentLength);

resp := innerRequest:GetResponse();
:RETURN resp:StatusCode;

:ENDPROC;

And then you create the GetFromAzureBlob to retrieve the file in a similar fashion:

:PROCEDURE GetFromAzureBlob;
:PARAMETERS remoteFile;

:DECLARE    sasToken, storageAccount, containerName, method, sampleContent,
            contentLength, requestUri, oWebService, oClient, oRequest, innerRequest,
            stream, resp, encoding, sTmpFileName;

encoding := LimsNetConnect("", "System.Text.Encoding",, .T.);


sasToken := "yourSAStoken";
storageAccount := "yourAzureAccount";
containerName := "yourAzureContainer";
blobName := remoteFile;

method := "GET";

requestUri := Replace(Replace(Replace(Replace(
                    "https://{storageAccount}.blob.core.windows.net/{containerName}/{blobName}?{sasToken}",
                        "{storageAccount}", storageAccount),
                        "{containerName}", containerName),
                        "{blobName}", blobName),
                        "{sasToken}", sasToken);

oWebService := WebServices{};
oClient := oWebService:CreateHttpClient();
oRequest := oClient:CreateHttpRequest(requestUri);

oRequest:Method := method;
oRequest:ContentType := "text/plain; charset=UTF-8";
resp := oClient:GetResponse(oRequest);
sTmpFileName := GlbDefaultTempDirectory + CreateGuid() + ".tmp";
resp:SaveValueToFile(sTmpFileName);
:RETURN sTmpFileName;

:ENDPROC;

Step 3- use it

As simple as that, you got yourself an upload and download to azure containers.

Conclusion

As you can see, as usual, this was quite easy! One just needs the correct information. Next step will be to see what more can containers bring to your STARLIMS installation.

Hope this can come in handy sometime to someone!


Identify Server Script Performance Bottlenecks

This week was Abbott Informatics’ APAC Forum. Speaking with old colleagues, I got inspired to try to revive this site for the 4th time (or is it the 5th?).

I’m currently sitting in a performance enhancement session and thinking to myself: heh, that’s NOT how I would go about it (sorry guys!). Silver bullets? Nah.

The first step to improving performances is to identify what is slow (duh!). What are the bottleneck? Why is it slow?

As a STARLIMS developer, I know that oftentimes, the code written in there is not necessarily the most efficient one. Therefore, why not start by monitoring the performances of, let’s say, SSL scripts, which represent the backbone of the business layer?

I’m thinking: why not have a simple tool that will record, like a stop watch, all block of code execution time, and then provide a report I can read? Heck, .NET has a StopWatch class! Hey! STARLIMS IS .Net!

The more I think about it, the more I consider: let’s do it!

How do we do this?

First, let’s create a class. I like classes. I like object-oriented code. I like the way it looks in SSL afterward, and it makes it way easier to scale through inheritance later on. Question is: what should the class do?

Well, thinking out loud, I think I want it to do is something like this:

  1. Start the stop watch
  2. Do something
  3. Monitor event 1
  4. Do something
  5. Monitor event 2
  6. Do something else
  7. Monitor event x
  8. so on and so forth
  9. Provide a readable report of all the event duration

I also want it to count the number of time an event run, and I want to know the AVERAGE time gone in there as well as the TOTAL time this event took.

Now that I know what I want, let’s write the class that will do it (for the sake of this example, I created it in the Utility global SSL category).

:CLASS perfMonitor;

:DECLARE oStopWatch;
:DECLARE nLastCheck;
:DECLARE aEvents;
:DECLARE WriteToLog;

:PROCEDURE Constructor;
:PARAMETERS bAutoStart;
:DEFAULT bAutoStart, .T.;
Me:WriteToLog := .F.;
Me:nLastCheck := 0;
Me:aEvents := { {"Event", "Total Duration", "# of calls", "Avg Duration"} };
Me:oStopWatch := LimsNetConnect("System", "System.Diagnostics.Stopwatch");
:IF bAutoStart;
	Me:Start();
:ENDIF;
Me:WriteToLog := .T.;
:ENDPROC;

:PROCEDURE Monitor;
:PARAMETERS sMessage;
:DEFAULT sMessage, "";
:DECLARE nElapsed, nDuration, i, bNew;
:IF Me:oStopWatch:IsRunning;
	Me:oStopWatch:Stop();
	nElapsed := Me:oStopWatch:ElapsedMilliseconds;
	nDuration := nElapsed - Me:nLastCheck;
	:IF Me:WriteToLog;
		UsrMes("Performance Monitor ==> " + LimsString(nDuration) + " ms. Message: " + sMessage);
	:ENDIF;
	Me:nLastCheck := nElapsed;
	bNew := .T.;
	:FOR i := 1 :TO Len(Me:aEvents);
		:IF Me:aEvents[i][1] == sMessage;
			Me:aEvents[i][2] += nDuration;
			Me:aEvents[i][3] += 1;
			Me:aEvents[i][4] := Me:aEvents[i][2] / Me:aEvents[i][3];
			bNew := .F.;
		:ENDIF;
	:NEXT;
	:IF bNew;
		aAdd(Me:aEvents, { sMessage, nDuration, 1, nDuration });
	:ENDIF;
	Me:oStopWatch:Start();
:ENDIF;
:ENDPROC;

:PROCEDURE Restart;
Me:Monitor("Internal Restart");
Me:oStopWatch:Restart();
:ENDPROC;

:PROCEDURE Stop;
Me:Monitor("Internal Stop");
Me:oStopWatch:Stop();
:ENDPROC;

:PROCEDURE Start;
Me:Monitor("Internal Start");
Me:oStopWatch:Start();
:ENDPROC;

:PROCEDURE ToString;
:RETURN BuildString2(Me:aEvents, CRLF, "	");
:ENDPROC;

The above gives us an object we can start, restart, and monitor events (messages). At the end, we use typical ToString() and will have our “report”. Example of using this:

:DECLARE oPerformanceMonitor;
oPerformanceMonitor := CreateUdObject("SGS_Utility.perfMonitor");
lWait(1.3); /* fake doing something that takes time;
oPerformanceMonitor:Monitor('Step 1');
lWait(0.8); /* fake doing something that takes time;
oPerformanceMonitor:Monitor('Step 2');
lWait(1.1); /* fake doing something that takes time;
oPerformanceMonitor:Monitor('Step 3');
lWait(1.45); /* call Step 1 again to generate an aggregate;
oPerformanceMonitor:Monitor('Step 1');
:RETURN oPerformanceMonitor:ToString();

If I run the above, the output will look like

Event	Total Duration	# of calls	Avg Duration
Step 1	2025	2	1012.5
Step 2	1015	1	1015
Step 3	1011	1	1011

I have been using this in many places in our system and it did help me to find the best places to optimize our code. Sometimes, the same insert will run 500 times and will total up to 15 seconds; that is worse than one call that runs only once and take 3 seconds (at least for the end user).

I hope this can help you find the bottlenecks of your SSL code!

c# Take a WebPage Snapshot

Alright, it’s been a while, and this time around, I have something good! I have this situation where I want to take a screenshot programmatically from a web page. Although there are many examples out there for thumbnails, none really matched what I needed. I wanted to provide:

  • A maximum Width
  • A maximum Height
  • URL
  • Header (in case you need a special authentication header)

Turns out that everything was out there, scattered pieces here and bits there.

So I tinkered the whole thing together, and came up with something that works actually quite good! I still have one challenge left though: how can I put that in a DLL I can reuse and not get a threading issue? That is the question.

Here’s the code – pretty much self explanatory.

/// <summary>
/// Take a snapshot of a web page. Image will be truncated to the smallest of 
/// - smallest between rendered width and maximum width
/// - smallest between rendered height and maximum heigth
/// </summary>
/// <param name="webUrl">URL to take a snapshot from</param>
/// <param name="authHeader">Authentication header if needed</param>
/// <param name="maxWidth">Maximum width of the screenshot</param>
/// <param name="maxHeight">Maximum height of the screenshot</param>
/// <param name="output">output image file name</param>
[STAThread]
static void WebPageSnapshot(string webUrl, string authHeader, int maxWidth, int maxHeight, string output)
{
    Uri uri = new Uri(webUrl);

    WebBrowser browser = new WebBrowser();
    browser.Size = new System.Drawing.Size(maxWidth, maxHeight);
    browser.ScrollBarsEnabled = false;

    browser.Navigate(uri, "", null, authHeader);
    
    // This is what will make this render completely
    while (browser.ReadyState != WebBrowserReadyState.Complete)
    {
        Application.DoEvents();
    }
    using (Bitmap bitmap = new Bitmap(width, height))
    {
        Point location = webBrowser.Location;
        webBrowser.DrawToBitmap(bitmap, new Rectangle(location.X, location.Y, webBrowser.Width, webBrowser.Height));
        bitmap.Save(output, ImageFormat.Png);
    }
}