How Libraries (DLL's) Work

Why Libraries Exist?

Let's say that you want to program your own application, an image viewer, for example. You need to be able to load image files and display them on the screen. That sounds simple enough.

The problem is that there are several image formats: PNG, JPEG, GIF, WebP, among many, many others. The algorithms you would need to write in your program's source code to decode the data of a PNG file into pixels you can display on the screen are rather complicated and easy to get wrong. It would be very easy to write an algorithm that can open some images right but then when you release your image viewer someone with millions of images will have an image that your algorithm fails to decode due to a mistake in your code (a bug).

So we don't do that.

Instead, we leave it in the hands of someone who knows how this stuff works and can write a bug-free program to do this common task. This developer publishes their PNG-decoding project as a library that other programs can use. The libpng.dll is an example of such library.

Then, instead of everyone writing their own PNG decoder, they can just use the code of that developer who is an expert in the field. This saves a lot of time and is a benefit for everyone.

Now, if a bug is found in how PNG is decoded, we tell the library developer that their code is bugged. If the bug gets fixed in the library, every single application that uses that PNG decoder benefits from it, so long as they grab the new version of the library with the bug patched.

Open Source Libraries

In many cases, a library is an open source library, meaning that its source code is available for everyone to read and modify. These are projects maintained by the community of developers. Since everyone benefits from having a library that opens files right, it makes sense that anyone can contribute to fixing bugs in this library that they would personally benefit from.

However, it's worth noting that not all libraries are open source. Closed-source libraries exist, and these are likely commercial libraries. In this case, the developer of an application must purchase a license to use the library.

In both cases, you get to use the code someone else made, which saves time.

Linking Types

In a desktop application, libraries can be used in two ways: with a dynamic link or with a static link.

Static Links

Statically linked libraries are embedded in the same executable file as the program, which means you don't have any separate .dll files.

It also means that a single library can't be shared between two applications since the library is built INTO the application now.

Dynamic Links

Dynamically linked libraries are libraries that aren't packaged in the same executable file as the main program, which means they need to be found and loaded somehow. How this works depends on the operating system and configurations.

For example, Windows will try to find any .dll files in the same folder as the .exe file and load them, as well as several other places, including the system32 folder. The search order is surprising complicated, to be honest. See [https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order] (accessed 2024-04-08). If a .dll required by the .exe isn't found, the operating system will display an error.

It's theoretically possible for a dynamically linked library that is used by multiple programs to be loaded in memory only once. In other words, when a program needs the library, the operating system loads it into memory, and then when another programs needs it, the operating system doesn't need to do anything because it's already loaded in memory. This only works if both programs require the exact same version of the library, and could save a bit of RAM and speed up loading applications. In practice, this doesn't happen on Windows since every single application comes with its own .dll's in its folder instead of depending on a shared .dll that may or may not exist somewhere else in the system.

Licenses Forbidding Static Linking

One reason why dynamic linking and thus .dll's are common is because many libraries used by programs are open source and have licenses that forbid statically linking such as the GPL.

The idea behind the GPL is that you, the user, can replace open source components of a application as you wish, and in this case it means you could, for example, replace the libpng.dll that comes with an image viewer by your own custom libpng.dll.

This could be useful of the version of the libpng.dll that an application uses doesn't work on your PC for some reason, and there is a compatible version that fixes the bug, but the application is abandonware so it's not getting any updates anymore. In this case, you could theoretically replace that library with the new version and fix the problem yourself.

However, if the library was statically linked, it would be inside the executable file, so you wouldn't be able to just replace the .dll file to fix your problem. That's what the GPL license forbids.

Javascript Libraries

Libraries aren't limited to desktop applications. Webpages, websites, and web applications can use libraries as well. These run a programming language called Javascript inside the web browser.

Javascript libraries are infamous for the fact that there are just way too many of them and a new one is created every other day. This is due to the web browser providing a lot of functionality, but yet not enough to make a lot of things.

For example, JQuery is library that was popular simply because for a whole decade some Javascript code only worked on some web browsers, so a library was necessary to provide the same functionality on all browsers. Nowadays, React is another popular library that provides a framework for building complex GUI's on the web browser.

Standard Library

Every programming language comes with something called a standard library which is built into the language. How large this standard library is varies from language to language.

To have an idea, a few common methods that we can find on standard libraries are about math, such as max(a, b) and min(a, b), which calculate the largest and smallest of two values, respectively, floor(a) and ceil(a), which round a number down and up, respectively, and geometric functions such as cos(x), sin(x), tan(x), etc.

Glue Code

It's important to understand that there are different types of programmers just as there are different types of programming projects.

In general, a library is a rather simple programming project. For example, if you have a library for encrypting data, it pretty much only needs to implement to subprograms: encrypt and decrypt. These can be very complicated algorithms to write that require a lot of expert knowledge, but the project itself is rather simple.

By contrast, applications are rather complex, despite their algorithms being very simple. For example, an image viewer application will use one library to display its windows, menubars, buttons, etc. This is called GUI toolkit, such as Qt or GTK. It will need a separate library to open image files. However, fortunately, these toolkits already come with their own image-loading sub-libraries (e.g. GdkPixbuf), which can handle a variety of common image formats. This means that the only thing the application code actually needs to do, and the only thing its developer needs to program, is gluing clicking on a button to displaying the file, and that's it.

On the other hand, there will be a separate subprogram for every single button you can click on. You need to glue File -> Open to displaying a file dialog. You need to glue File -> Exit to closing the window. You need to glue pressing the next button to showing the next image. The algorithm for each of these actions is much simpler than the algorithm to encrypt and decrypt data, but there is just far more actions in an application, so it's far more places where a mistake can occur.

Note that most of the code of the programs that are being executed in an application weren't written by the application developer but by the developers of the libraries, which makes it very difficult for the application developer to actually understand everything that is going on in their application. They haven't written the code that loads the images, or that displays windows, they have simply glued one thing to another.

This is the main reason why some very bad programmers can create some very cool programs. You see this a lot in games these days. Anyone can create an extremely unoptimized 3D game in Godot or Unity. This is really cool, because the alternative would be that these games wouldn't be created, so third-party libraries enabled these terrible programmers to do things they otherwise wouldn't be able to do. I assume AI is going to do the same.

Additionally, another issue is how to test these two different types of projects.

Automated tests are programs written by a programmer to test another program. These ensure each subprogram functions as it should, and if all subprograms are working, then the whole program should work as well.

A library is simple, so it's very easy to test. Most of its subprograms could be pure functions, meaning that the output of the function depends only on its input parameters, and doesn't modify the state, i.e. if the subprogram is run twice with the same parameters, you always get the same output.

By contrast, even the simplest application will have stateful subprograms. For example, take Help -> About. It displays the about dialog. However, more specifically, it only displays an about dialog if the about dialog isn't currently open; if it's already open, it should raise the about dialog so it's visible. This means a test for this subprogram would be more complicated than a test for encrypt and decrypt, since we have to account for the state of the application. Note that in most applications, a lot of the interface is disabled if you don't have a file open, or some operations have different behavior if a new file hasn't been saved yet. This is a type of complexity that libraries simply don't have to deal with.