Of all the modern, cross-platform GUI toolkits, the only I like the most is Qt, because it's a proper desktop UI toolkit and it's not called GTK. However, there are a few problems: first of all, I don't like C++. Having to write the same code in both a .hpp header file and in a .cpp source file is painful, specially since they are not connected in any standard way, so tooling often has issues, specially considering that preprocessors can just inject anything in any file so in order for any tool to parse C++ (or C) it MUST first handle all #include and other macro definitions that manipulate the source code. Compile times always make me feel sad, specially as the compile times keep growing as a project complexity keeps growing, and any GUI project already starts extremely bloated with all the input and graphics code it needs. And I don't like all the memory errors I often get and having to make structural decisions about which type of pointer do I have to use in a function. That's why I like programming Qt in Python instead, since Python solves all of these issues: good tooling, no compile times, memory managed language. But Python also has its own problem: it's slow. For performance-critical applications, or, honestly, just any resource-intensive program, it would be better if the program was written in a low-level language instead. There is a low-level language that solves the issues of C++ AND of Python: Rust. Rust is a low-level memory safe programming language, but it isn't C++, which means that in order to use Qt in Rust, you need a Qt binding for Rust. There is no official Qt binding for Rust right now. Even if there was, the compile times would still be very high. Fortunately, there are official bindings for Python, such as PySide6 for Qt6, for example. One way to solve these problems would, then, be to create the GUI in Python, somehow make Python tell Rust to render critical parts in a pixel buffer, pass a pointer to the pixel buffer back to Python, then make Python pass the pointer to Qt, so that Qt can render it on the screen. In other words, make Python become the bridge between Rust and Qt so you don't need Qt bindings in Rust just to do math faster.
This is rather complicated, so I'll only write the higher-level concepts you need to be aware of. I might release the source code of an example of how to do this later.
Making Rust Accessible from Python with PyO3/Maturin
In order to do these things, the first thing we need to do is to compile the Rust code in a way it can be used by Python. To do this, we compile Rust as a "native Python module," which is essentially a library that the Python interpreter can load.
To generate the native Python module from Rust code, we need to use a tool like Maturin by PyO3. Most of this article is just a paraphrase of what is already written in the official tutorial.
This tool is a build system that you define in your pyproject.toml and install via pip. When you run the command maturin develop in the terminal, it runs cargo build to compile the native Python module (that will have .so extension on Linux, and .dll on Windows), and place it inside your Python's source code directory. You'll need to update your .gitignore to avoid committing the compiled code to your Git repository, by the way. In order for this to work, you must annotate the Rust functions with #[pyclass], #[pymethods] and so on. And write a #[pymodule] function that adds the classes that you want to export to Python to a module object. Your #[pymodule] must have the same name as the module that you want to export. For example, if your project Python module is called myproject, and you want your Rust module to live in myproject.crab, then the function annotated by #[pymodule] must be called crab.
// place this in the mod.rs of your Python-facing module for example,
// e.g. in a myrustmodule/mod.rs
pub mod mystruct;
use pyo3::prelude::*;
use mystruct::StructIWantToExport;
// name of this function must match the module's name last segment
#[pymodule]
fn crab(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add_class::<StructIWantToExport>()?;
Ok(())
}Fixing the Autocomplete by Generating Stubs
Although this will make the methods of StructIWantToExport available in the Python interpreter, it will be a bit difficult to use it in Python code since your IDE will not know about what members the crab module has. That is, it will RUN, but during development you won't see StructIWantToExport after typing myproject.crab..
In order to fix this, you need to generate Python stubs. This will require adding more annotations via pyo3-stub-gen, such as #[gen_stub_pyclass] and #[gen_stub_pymethods].
You'll also need to use the define_stub_info_gatherer! macro to define a function that gathers all the annotations. This function will be called, for example, stub_info. Then you'll need to create a bin/stub_gen.rs with a main function that will call this function to generate the stubs. This means that after you run maturin develop, you will also need to run cargo run --bin stub_gen to run the function.
use pyo3_stub_gen::Result;
fn main() -> Result<()> {
// `stub_info` is a function defined by `define_stub_info_gatherer!` macro.
let stub = myproject::myrustmodule::stub_info()?;
stub.generate()?;
Ok(())
}An Issue with Rust-Analyzer in VS Code
If you are using the rust-analyzer extension in VS Code, you'll run into a problem where everything is going to recompile all the time because rust-analyzer and maturin call cargo check and cargo build with different parameters, invalidating the cache every time either is run.
I'm not sure how to fix this properly yet, but a quick workaround is to set "rust-analyzer.cargo.targetDir": "target-rustanalyzer" in .vscode/settings.json. This will make the rust-analyzer extension use a different target directory (called target-rustanalyzer) so the cache won't be invalidated. It does end up using twice as much space, though.
Creating a Pixel Buffer in Rust
To create a pixel buffer in Rust, we need to define a simple struct for our RGBA colors and create a Vec whose length equals the width times the height of the buffer.
#[derive(Copy, Clone, Debug, Default, Eq, PartialEq, Ord, PartialOrd, Hash)]
// repr(C) forces Rust to align the r, g, b, a fields the same
// way that C does, which is important since we need them to be in the
// correct order for when we pass pointers around.
#[repr(C)]
pub struct RgbaColor {
pub r: u8,
pub g: u8,
pub b: u8,
pub a: u8,
}
pub fn make_buffer(width: usize, height: usize) -> Vec<RgbaColor> {
vec![RgbaColor { r: 0, g: 0, b: 0, a: 0 }; width * height];
}You can either implement all the drawing functions yourself or use a library that works in buffers in this format. There are also other color formats that Qt supports, by the way, but for this article we'll use RGBA, and not ARGB or anything else.
Passing a Pointer to the Pixel Buffer to Python
there are various ways to pass the pixel buffer from Rust to Python, but the only way I found that does so without copying the entire buffer is using a MemoryView. That is, if you use Python Bytes, you can copy the whole buffer and then the copy is owned by the Python interpreter. With a MemoryView, you pass the buffer in a zero-copy way. This means that the Rust side still owns the pointer, but there are no longer memory safety guarantees.
/**
* writable_within_python sets a flag in the Python's MemoryView object that can prevent
* writes to the vector WITHIN Python. If you take the MemoryView and use it to pass the
* pointer to a non-Python library, the flag may not be honored. In particular, in Qt you
* can create a QPainter to draw on QImage constructed from a "read-only" MemoryView and
* doing so WILL write to the "read-only" array. PySide6 doesn't check if the MemoryView
* is supposed to be read-only or not.
*/
pub fn get_memory_view_from_vector<'a, T>(py: Python<'a>, vector: &'a Vec<T>, writable_within_python: bool) -> PyResult<Bound<'a, PyAny>> {
let byte_count = vector.len() * mem::size_of::<T>();
let mem_ptr;
let flags = match writable_within_python {
true => PyBUF_READ | PyBUF_WRITE,
false => PyBUF_READ
};
unsafe {
mem_ptr = PyMemoryView_FromMemory(vector.as_ptr() as *mut c_char, byte_count as isize, flags);
}
if mem_ptr.is_null() {
return Err(PyErr::fetch(py));
}
unsafe {
Ok(Bound::from_owned_ptr(py, mem_ptr).cast_into_unchecked())
}
}There are a few important issues to take note of.
First off, we are simply passing a raw pointer to Python, which means we still HAVE to drop the Vec from the Rust side. Most importantly, not only we "have" to (since Rust does it automatically), but we "CAN" drop the Vec from the Rust side. This means you must manually manage the lifetime of the MemoryView from the Python side. If the Vec is dropped in Rust, but the MemoryView still lives in Python, we will be able to access memory from Python that is no longer allocated or contains garbage.
Second off, keep in mind that we are not passing a pointer to the Vec directly, but a pointer to the array managed by the Vec. If we add elements to the Vec with push that go beyond the array's capacity, the Vec will be resized which likely means the memory address of the array will change. The way structures like a Rust's Vec wors is that it allocates a space of contiguous memory for its internal array, and if that space isn't sufficient anymore, it just allocates a different, bigger space of contiguous memory. You would think that it should be able to tell the operating system "can you just make the space I got bigger, and I keep the same memory address?" but that wouldn't be possible if the space right after is already reserved by some other part of the program. That's why changing a Vec's size generally invalidates its internal pointer.
Fortunately, this is a pixel buffer, so we probably won't be adding items with push. However, there may be cases we want to resize the dimension (width and height), which will change the internal array. Or we may simply want to drop that Vec because we don't have any use for it anymore, e.g. it won't be rendered anymore. In which case the pointer will become invalid.
You may think that this won't be a problem: just ensure that you stop using the pointer in your Python code after you drop the object that owns the pointer from the Rust code. However, if you pass the pointer to a library, you must ensure that the library doesn't keep a reference to the pointer either. For example, in Qt, you can "copy" a QImage in the "Ctrl+C" sense with the clipboard API. Doing this will give a reference to the QImage to the clipboard. Normally, the QImage owns the pixel buffer, so when its reference count become zero the data is automatically deleted. However, if the QImage does NOT own the pixel buffer, and you delete the pixel buffer first, then the clipboard will keep a reference to deleted memory. In this case, you must create a copy of the QImage using QImage.copy().
Third off, a MemoryView has a flag that lets us make it read-only. One would think that this flag lets us pass immutable data to Python, but there is a caveat: the flag is only honored within code that uses the MemoryView API. If we use the MemoryView to pass a raw pointer from Rust to C++, the C++ code probably won't be aware that the data was supposed to be read-only, so it may write to it. This is the case with Qt. If you create a QImage from a MemoryView, you can always paint on it using a QPainter, modifying the bytes even when the MemoryView is flagged as read-only.
Now that we know this, let's see how we can actually use the MemoryView from Python.
Rendering a MemoryView in Python with Qt
With the code below, we can see how we can use the Vec in Python with zero-copy. We still end up copying the pixel data to render it, but it's one copy fewer, which is faster.
from PySide6.QtGui import QImage, QPainter
from PySide6.QtWidgets import QWidget
from . import crab
class MyWidget(QWidget):
def __init__(self, event: QPaintEvent):
self.struct_that_renders_images = crab.StructThatRendersImage()
def paintEvent(self, event: QPaintEvent):
width, height = self.width(), self.height()
# this function creates a buffer of width x height size
# and stores it as a Vec inside struct_that_renders_images
self.struct_that_renders_images.render(width, height)
# get the pointer to the struct's Vec's internal array
rgba_bytes = self.struct_that_renders_images.get_memory_view_for_my_buffer()
# zero-copy QImage from pointer
bytes_per_line = width * 4
image = QImage(rgba_bytes, width, height, bytes_per_line, QImage.Format.Format_RGBA8888)
# begin drawing on this QWidget
painter = QPainter(self)
# copies pixels from buffer
painter.drawImage(0, 0, image, 0, 0, image.width(), image.height())
# reference to image is dropped and pixels were already copied
# so memory safety is ensuredRendering to a MemoryView in Python with Qt
It's also possible to use the same idea to render on the Vec using any of the QPainter functions just like we could with any QImage.
class MyWidget(QWidget):
def __init__(self, event: QPaintEvent):
self.struct_that_renders_images = crab.StructThatRendersImage()
# initialize buffer size
self.struct_that_renders_images.set_size(600, 400)
self.drawn_text_centered("Hello World!")
def draw_text_centered(self, text: str):
width, height = self.struct_that_renders_images.get_size()
rgba_bytes = self.struct_that_renders_images.get_memory_view_for_my_buffer()
# zero-copy QImage from pointer
bytes_per_line = width * 4
image = QImage(rgba_bytes, width, height, bytes_per_line, QImage.Format.Format_RGBA8888)
# begin drawing on the image
painter = QPainter()
painter.begin(image)
try:
painter.setRenderHint(QPainter.RenderHint.Antialiasing)
painter.setRenderHint(QPainter.RenderHint.TextAntialiasing)
font = QFont("Arial", 24, QFont.Weight.Bold)
painter.setFont(font)
painter.setPen(QColor("black"))
image_rect = image.rect()
painter.drawText(image_rect, Qt.AlignmentFlag.AlignCenter, text)
finally:
painter.end()
The same idea can be employed on any Python graphics library that can use a MemoryView. For example, if there was a Python graphics library that rendered 3D objects, and you wanted to display those in a Qt GUI, but you wanted to add some more resource-intensive graphics on top or below of those 3D objects, you could simply use both the library API and your own Rust module with the same buffer. Although it probably makes more sense to use some sort of layer system for this instead.
Python as a Qt Binding?
While I was programming this I had a brilliant idea. PyO3 essentially lets you program the Python interpreter from Rust. That means it allows us to execute Python commands directly from Rust. What if, instead of using Qt bindings for Rust, we simply imported PySide6 in the Python interpreter from the Rust side and called some functions on it? That sounds way more difficult than just writing Python code, but it should be technically possible!
On Refactoring
It's important to note that if you use a mixed-language project like this, refactoring an interface in one language isn't reflect to the other language, which can lead to subtle bugs if you use a duck-typed language like Python without a linter.
If we rename a method in Rust, rust-analyzer will update all references within the Rust part of the code base. But the Python part that uses the exported method will not be updated. When we regenerate the native Python module with maturin develop, it will just change the module's structure leaving any Python code that used the old name in a broken state.
The same would happen if we were calling Python code from Rust.
It's probably a better idea to keep the amount of exposed interfaces to a minimum. Instead of exporting every single Rust object to Python, only export a few high-level structs whose responsibility is to translate Python to Rust.
For example, if you have a window that can open N documents, and those documents have Rust objects. And you want a Document struct in Rust to be the source of truth for a property like filepath, but there are other properties that are only used by Rust code. You don't need to export the entire Document struct as Python class. You could simply have a RustWindow class that has something like get_document_filepath(&self, index: usize), because that's the only information that you actually need. If you don't like index you can replace it with simple stable opaque unique identifier that you generate inside RustWindow every time you add a new document.