SQL Server on Linux: Behind The Scenes

Now it has become known that in order to make SQL Server run on Linux one has to introduce the so-called Platform Abstraction Layer (PAL) into SQL Server. The PAL is used to align all platform or operating system specific code in a single place and by doing so allow the rest of the codebase to stay operating system agnostic. The PAL was never previously required because SQL Server has been running only on Windows operating system for a long time. In reality, to ensure that numerous functionalities are available the SQL Server database engine codebase contains plenty of references to popular libraries on Windows.

When bringing SQL Server to Linux the IT giant Microsoft made sure to bring to Linux the performance, scale value and the full functional of the SQL Server RDBMS. And by doing that they made sure that the applications that performed well in SQL Server on Windows are equally good in SQL Server on Linux. For that purpose, Microsoft development team made a decision to marry parts of the Microsoft Research (MSR) project Drawbridge with the existing platform layer of SQL Server called SQL Server Operating System (SOS) with the goal of creating the SQLPAL that ultimately made possible the well-crafted SQL Server outside Windows.

As far as the Drawbridge project is concerned, it has provided abstraction between the underlying operating system and the application which ensured the secure containers, while SOS provided robust memory management, IO services and thread scheduling. Through the creation of SQLPA Microsoft developers made possible to use the existing Windows dependencies on Linux with the help of parts of the Drawbridge design focused on OS abstraction while leaving to SOS the key OS services. Moreover, they have changed the SQL Server database engine code to call directly into SQPAL for resource intensive functionality instead of Windows libraries.

Model of the Process

The diagram below demonstrates what the address space looks like when running. Here, the host extension is merely a native Linux application. Upon starting the host extension loads and initializes SQLPAL, SQLPAL and then brings up SQL Server. In addition, the team made sure that SQLPAL is able to launch software isolated processes that are merely a collection of threads and allocations running within the same address space. This is used for such things as SQLDumper, an application that is run when SQL Server encounters a problem to collect an enlightened crash dump.

Yes, at first look some users may think that this diagram shows a lot of layers, but in reality, there are no hard boundaries between SQL Server and the host.

Sql On Linux: Process Model

Advancement of SQLPAL

When the project began, SQL Server was built on SOS and Library OS was independent. The ultimate goal set by Microsoft team is to have a merged SOS and Library OS as the core of SQL PAL.  When it was released initially for public preview, this merge wasn’t fully completed, but the heart of SQLPAL had been replaced with SOS.  For instance, instead of the original Drawbridge implementations the threads and memory already use SOS functionality.

The outcome of that is that there are two instances of SOS running inside the CTP1 release: one in SQL Server and one in SQLPAL. The reason it still works perfectly fine is that the SOS instance in SQL Server is still using Win32 APIs which call down into the SQLPAL. And instead of Win32 now SQLPAL instance of the SOS code calls the host extension ABIs (i.e. the native Linux code).

Currently, Microsoft is working on removing the SOS instance from SQL Server by exposing the SOS APIs from the SQLPAL. When this is going to be completed is still unclear but as soon as they do it everything will flow through the single SQLPAL SOS instance.

Leave a Reply

Your email address will not be published. Required fields are marked *