In the previous part of this series, we discussed why we opted for Realm to create databases that back up complex mobile applications and its varied use cases. We also discussed, what were our application performance requirements, and how Realm would be able to fulfill those most efficiently. Its simplicity, speed, and mobile-first design just won out.
In this part, we take the next step, looking into aspects of the Realm core and how it functions at the engine level so that we can leverage its maximum potential and set it up in a way that delivers the best results for the application. We'll also delve into the decisions that Realm builders took to ensure that complex applications built using Realm enable lightening-fast decision-making and are high-performing.
Now onto the fun stuff, the technical amazingness:
When we use object-relational mapping (ORM), there’s a lot of abstractions that come with it. Realm chose to build their core engine from scratch — to make the bindings thinner, to remove dependencies and complexities of building, and layering over existing database engines.
This engine has some other really intriguing elements.
● Multiversion concurrency control (MVCC): For read/write locking, Realm follows an MVCC approach. Think of Realm's internal model to be a lot like Git with concepts like branches and atomic commits. We can think of every transaction as a snapshot of the database when the transaction starts. This is why we can make a bunch of changes with hundreds of threads concurrently, accessing the database without introducing errors. Writes are performed as append-only— i.e. we have an immutable view of data, therefore allowing us to write, without blocking the read.
● Internal structure: The entire structure inside is a giant B+ tree. The to-one & to-many relationships are treated as first class citizens. No expensive joint operations are performed. This is great for object graph traversals, easily following the pointers.
● Optimization for string and integers (Int.): Realm creates common string values as enums, which help with quick lookups. This operation is a little expensive, but it ensures smaller file sizes and faster reads.
Integer packing is done, so ints take as little space as possible. It doesn't matter if you specify it as 8, 16, 32, or 64. Under the hood, Realm stores it in the most optimized way it can, with little to no performance overhead.
● Data safety for crashes: Sudden crashes or app. failures shouldn't corrupt your database file. Since Realm's internals are like Git and it is a giant B+ tree, at any point in time, you have the top level commit (HEAD commit). So the pointer will only move after a two-phase commit, once it ensures that the data is written to the disk safely.
The worst that can happen in a write transaction is you’ll lose just the parts that you were working on and not the whole thing.
● Zero-copy: Traditional ORMs must copy data. Data is initially on a disk; first you read from disk, then copy raw data into deserialized intermediate in memory representation (memory allocation), and then copy the intermediate representation into language level in-memory object (memory allocation).
Realm skips the entire copy part. The entire Realm file is memory mapped and of the same format as that on disk. Realm creators took this into consideration for the core file format. They made sure that the format on disk is readable in memory, making deserialization unnecessary. The only thing remaining is to calculate and read data from the offset and return the raw value from property access.
● Encryption (built-in): Due to the way realm stores the database file, it's very easy to build encryption. As mentioned earlier, the entire file is memory mapped so the encryption can be built just like in Linux systems. All cryptography is done in virtual memory. It marks memory as read-write protected and it can throw access violations.
Here are some examples of writing Realm queries, and creating Realm instances, and updates.
This is just the tip of the iceberg. Realm offers vast querying and writing options for our data. For further readings we can refer to the official docs.
Realm also provides a syncing facility with backend-deployed servers.
There is another variant of Realm for special purposes, in which all access to data and/or queries are done using string-based class names instead of class-type references.
This is the Dynamic Realm. This is useful during migrations or when working with string-based data like CSV or XML files. But, that’s another story for another time.
Happy coding!
.png)
.png)
.svg)
.svg)
.svg)
.png)
.png)
.png)
%20(2).svg)
.svg)
.jpg)
.svg)
In the previous part of this series, we discussed why we opted for Realm to create databases that back up complex mobile applications and its varied use cases. We also discussed, what were our application performance requirements, and how Realm would be able to fulfill those most efficiently. Its simplicity, speed, and mobile-first design just won out.
In this part, we take the next step, looking into aspects of the Realm core and how it functions at the engine level so that we can leverage its maximum potential and set it up in a way that delivers the best results for the application. We'll also delve into the decisions that Realm builders took to ensure that complex applications built using Realm enable lightening-fast decision-making and are high-performing.
Now onto the fun stuff, the technical amazingness:
When we use object-relational mapping (ORM), there’s a lot of abstractions that come with it. Realm chose to build their core engine from scratch — to make the bindings thinner, to remove dependencies and complexities of building, and layering over existing database engines.
This engine has some other really intriguing elements.
● Multiversion concurrency control (MVCC): For read/write locking, Realm follows an MVCC approach. Think of Realm's internal model to be a lot like Git with concepts like branches and atomic commits. We can think of every transaction as a snapshot of the database when the transaction starts. This is why we can make a bunch of changes with hundreds of threads concurrently, accessing the database without introducing errors. Writes are performed as append-only— i.e. we have an immutable view of data, therefore allowing us to write, without blocking the read.
● Internal structure: The entire structure inside is a giant B+ tree. The to-one & to-many relationships are treated as first class citizens. No expensive joint operations are performed. This is great for object graph traversals, easily following the pointers.
● Optimization for string and integers (Int.): Realm creates common string values as enums, which help with quick lookups. This operation is a little expensive, but it ensures smaller file sizes and faster reads.
Integer packing is done, so ints take as little space as possible. It doesn't matter if you specify it as 8, 16, 32, or 64. Under the hood, Realm stores it in the most optimized way it can, with little to no performance overhead.
● Data safety for crashes: Sudden crashes or app. failures shouldn't corrupt your database file. Since Realm's internals are like Git and it is a giant B+ tree, at any point in time, you have the top level commit (HEAD commit). So the pointer will only move after a two-phase commit, once it ensures that the data is written to the disk safely.
The worst that can happen in a write transaction is you’ll lose just the parts that you were working on and not the whole thing.
● Zero-copy: Traditional ORMs must copy data. Data is initially on a disk; first you read from disk, then copy raw data into deserialized intermediate in memory representation (memory allocation), and then copy the intermediate representation into language level in-memory object (memory allocation).
Realm skips the entire copy part. The entire Realm file is memory mapped and of the same format as that on disk. Realm creators took this into consideration for the core file format. They made sure that the format on disk is readable in memory, making deserialization unnecessary. The only thing remaining is to calculate and read data from the offset and return the raw value from property access.
● Encryption (built-in): Due to the way realm stores the database file, it's very easy to build encryption. As mentioned earlier, the entire file is memory mapped so the encryption can be built just like in Linux systems. All cryptography is done in virtual memory. It marks memory as read-write protected and it can throw access violations.
Here are some examples of writing Realm queries, and creating Realm instances, and updates.
This is just the tip of the iceberg. Realm offers vast querying and writing options for our data. For further readings we can refer to the official docs.
Realm also provides a syncing facility with backend-deployed servers.
There is another variant of Realm for special purposes, in which all access to data and/or queries are done using string-based class names instead of class-type references.
This is the Dynamic Realm. This is useful during migrations or when working with string-based data like CSV or XML files. But, that’s another story for another time.
Happy coding!
In the previous part of this series, we discussed why we opted for Realm to create databases that back up complex mobile applications and its varied use cases. We also discussed, what were our application performance requirements, and how Realm would be able to fulfill those most efficiently. Its simplicity, speed, and mobile-first design just won out.
In this part, we take the next step, looking into aspects of the Realm core and how it functions at the engine level so that we can leverage its maximum potential and set it up in a way that delivers the best results for the application. We'll also delve into the decisions that Realm builders took to ensure that complex applications built using Realm enable lightening-fast decision-making and are high-performing.
Now onto the fun stuff, the technical amazingness:
When we use object-relational mapping (ORM), there’s a lot of abstractions that come with it. Realm chose to build their core engine from scratch — to make the bindings thinner, to remove dependencies and complexities of building, and layering over existing database engines.
This engine has some other really intriguing elements.
● Multiversion concurrency control (MVCC): For read/write locking, Realm follows an MVCC approach. Think of Realm's internal model to be a lot like Git with concepts like branches and atomic commits. We can think of every transaction as a snapshot of the database when the transaction starts. This is why we can make a bunch of changes with hundreds of threads concurrently, accessing the database without introducing errors. Writes are performed as append-only— i.e. we have an immutable view of data, therefore allowing us to write, without blocking the read.
● Internal structure: The entire structure inside is a giant B+ tree. The to-one & to-many relationships are treated as first class citizens. No expensive joint operations are performed. This is great for object graph traversals, easily following the pointers.
● Optimization for string and integers (Int.): Realm creates common string values as enums, which help with quick lookups. This operation is a little expensive, but it ensures smaller file sizes and faster reads.
Integer packing is done, so ints take as little space as possible. It doesn't matter if you specify it as 8, 16, 32, or 64. Under the hood, Realm stores it in the most optimized way it can, with little to no performance overhead.
● Data safety for crashes: Sudden crashes or app. failures shouldn't corrupt your database file. Since Realm's internals are like Git and it is a giant B+ tree, at any point in time, you have the top level commit (HEAD commit). So the pointer will only move after a two-phase commit, once it ensures that the data is written to the disk safely.
The worst that can happen in a write transaction is you’ll lose just the parts that you were working on and not the whole thing.
● Zero-copy: Traditional ORMs must copy data. Data is initially on a disk; first you read from disk, then copy raw data into deserialized intermediate in memory representation (memory allocation), and then copy the intermediate representation into language level in-memory object (memory allocation).
Realm skips the entire copy part. The entire Realm file is memory mapped and of the same format as that on disk. Realm creators took this into consideration for the core file format. They made sure that the format on disk is readable in memory, making deserialization unnecessary. The only thing remaining is to calculate and read data from the offset and return the raw value from property access.
● Encryption (built-in): Due to the way realm stores the database file, it's very easy to build encryption. As mentioned earlier, the entire file is memory mapped so the encryption can be built just like in Linux systems. All cryptography is done in virtual memory. It marks memory as read-write protected and it can throw access violations.
Here are some examples of writing Realm queries, and creating Realm instances, and updates.
This is just the tip of the iceberg. Realm offers vast querying and writing options for our data. For further readings we can refer to the official docs.
Realm also provides a syncing facility with backend-deployed servers.
There is another variant of Realm for special purposes, in which all access to data and/or queries are done using string-based class names instead of class-type references.
This is the Dynamic Realm. This is useful during migrations or when working with string-based data like CSV or XML files. But, that’s another story for another time.
Happy coding!
.svg)
.svg){kind=small}
.png){kind=small}
