Using Field Attributes with serde.rs

Using Field Attributes with serde.rs

Serde is a generic framework for serializing and deserializing Rust data structures efficiently.

use serde::{Serialize, Deserialize};

/// Represents the Discord information as
/// returned from the API
///
#[derive(Debug, Serialize, Deserialize)]
pub struct DiscordUser {
    pub photo_url: String,
    pub username: String,
    pub id: String,
}

Frequently associated with JSON serializing and deserializing, serde.rs can actually handle a wide range of formats:

The following is a partial list of data formats that have been implemented for Serde by the community (list copied from serde.rs documentation).

  • JSON, the ubiquitous JavaScript Object Notation used by many HTTP APIs.

  • Bincode, a compact binary format used for IPC within the Servo rendering engine.

  • CBOR, a Concise Binary Object Representation designed for small message size without the need for version negotiation.

  • YAML, a self-proclaimed human-friendly configuration language that ain’t markup language.

  • MessagePack, an efficient binary format that resembles a compact JSON.

  • TOML, a minimal configuration format used by Cargo.

  • Pickle, a format common in the Python world.

  • RON, a Rusty Object Notation.

  • BSON, the data storage and network transfer format used by MongoDB.

  • Avro, a binary format used within Apache Hadoop, with support for schema definition.

  • JSON5, a superset of JSON including some productions from ES5.

  • Postcard, a no_std and embedded-systems friendly compact binary format.

  • URL query strings, in the x-www-form-urlencoded format.

  • Envy, a way to deserialize environment variables into Rust structs. (deserialization only)

  • Envy Store, a way to deserialize AWS Parameter Store parameters into Rust structs. (deserialization only)

  • S-expressions, the textual representation of code and data used by the Lisp language family.

  • D-Bus’s binary wire format.

  • FlexBuffers, the schemaless cousin of Google’s FlatBuffers zero-copy serialization format.

  • Bencode, a simple binary format used in the BitTorrent protocol.

  • DynamoDB Items, the format used by rusoto_dynamodb to transfer data to and from DynamoDB.

  • Hjson, a syntax extension to JSON designed around human reading and editing. (deserialization only)

As you can see the list of data formats accepted can be quite extensive.

But what exactly is Serialization and Deserialization?

Well, in computing, serialization is the process of translating a data structure or object state into a format that can be stored (for example, in a file system or memory data buffer) or transmitted (for example over a network) and able to reconstructed later.

The process of serializing an object is also called marshalling an object in some situations. As you can imagine the opposite operation, extracting a data structure from a series of bytes, or doing the reverse process, is deserialization, also called unmarshalling.

Ok! Now that we got the basics out of the way, let’s find out how to use a serde.rs Field Attribute to solve a real problem.

Let’s start with a very basic example. In this example we will be working with a JSON payload, that not only is human readable and easier to understand but also probably one of the most common use cases in the day to day of a developer.

Here we have a really basic example, where we create a object based on a struct type, serialize it to a string and then deserialize it from the string. Pretty straight forward interaction I would say.

use serde::{Serialize, Deserialize};

/// The `Person` struct is the entry point into the program.
#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    age: u32,
}

fn main() {
    println!("Hello, serde.rs Field Attributes Example!");

    // Create a `Person` struct.
    let person = Person { name: "Josh".to_string(), age: 32 };
    println!("Person: {:?}", person);

    // Serialize the `Person` struct to a string.
    let serialized = serde_json::to_string(&person).unwrap();
    println!("Person Serialized: {}", serialized);

    // Deserialize the `Person` struct from a string.
    let deserialized:Person = serde_json::from_str(&serialized).unwrap();
    println!("Deserialized: {:?}", deserialized);
}

When you run you should get something like this:

Hello, serde.rs Field Attributes Example!
Person: Person { name: "Josh", age: 32 }
Person Serialized: {"name":"Josh","age":32}
Deserialized: Person { name: "Josh", age: 32 }

Great right! Where could this go wrong? Well, in this scenario not much could go wrong because you are controlling all the parts, you are creating an object with all the necessary data and you know you have all you need. But in a real life project you would probably receive a JSON payload as a response from some request, where-ever that may be.

But what if we change this code a bit to simulate a piece of the information missing when we try to deserialize it, or as you may also know it, unmarshall the data.

Example:

use serde::{Serialize, Deserialize};

/// The `Person` struct is the entry point into the program.
#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    age: u32,
}

fn main() {
    println!("Hello, serde.rs Field Attributes Example!");

    let serialized = "{\"name\":\"Josh\"}".to_string();

    // Deserialize the `Person` struct from a string.
    let deserialized:Person = serde_json::from_str(&serialized).unwrap();
    println!("Deserialized: {:?}", deserialized);
}

In this example we got rid of the serializing from a struct to string part, because we only want to focus on the deserializing part, simulating a JSON payload that would arrive from somewhere else. As you can see, our payload is missing one value, the age , so let’s see what happens when we run this code.

Hello, serde.rs Field Attributes Example!
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `age`", line: 1, column: 15)', src/main.rs:16:65
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

WOW! What happened?! Well, as you can probably see from the error message, the main thread panicked because tried initializing our struct object with a missing value, in this case age . But what can we do here?

Some programming language, more high level language could take of this automatically for us, by maybe omitting the value all together or just assigning a zero value for missing properties, but Rust will not do that out of the box for us, we need to tell it what to do in this case.

We have two options here:

The first approach and more straight forward and simple is to just tell serde to use the Default::default() value for a specific property that might be missing. Here’s how that would work.

use serde::{Serialize, Deserialize};

/// The `Person` struct is the entry point into the program.
#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    #[serde(default)]
    age: u32,
}

fn main() {
    println!("Hello, serde.rs Field Attributes Example!");

    let serialized = "{\"name\":\"Josh\"}".to_string();
    println!("Serialized: {}", serialized);

    // Deserialize the `Person` struct from a string.
    let deserialized:Person = serde_json::from_str(&serialized).unwrap();
    println!("Deserialized: {:?}", deserialized);
}

In the above code you can see that we add the Field Attribute “default” in the property age . By doing this we are telling serde to use the Default::default() value for the property, in case the value is not present.

When you run you should get something like this:

Hello, serde.rs Field Attributes Example!
Serialized: {"name":"Josh"}
Deserialized: Person { name: "Josh", age: 0 }

Ok, this works fine, but what if you have other plans for the default value of a property. For instance, for sake of argument, let’s say that we only count age in integers so a baby for example could have 0 years until is turns 1. But if you receive a payload with a age missing, you do not want the Default value of an integer assigned to it, because it could mean it is a new born baby with 0 years. So let’s say that in this case you want to assign a -1 to a missing age value.

This is how I would do it:

use serde::{Serialize, Deserialize};

/// The `Person` struct is the entry point into the program.
#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    #[serde(default = "missing_value")]
    age: i32,
}

/// The `missing value` function is called to get the default value for the field `age`.
fn missing_value() -> i32 {
    -1
}

fn main() {
    println!("Hello, serde.rs Field Attributes Example!");

    let serialized = "{\"name\":\"Josh\"}".to_string();
    println!("Serialized: {}", serialized);

    // Deserialize the `Person` struct from a string.
    let deserialized:Person = serde_json::from_str(&serialized).unwrap();
    println!("Deserialized: {:?}", deserialized);
}

Let me explain what is going on here. If we get a payload with the value of age missing, instead of just assigning Default::default() , serde will call the function missing_value to get the default value for that property. In this case we are returning -1 .

So when we run this code we should get this:

Hello, serde.rs Field Attributes Example!
Serialized: {"name":"Josh"}
Deserialized: Person { name: "Josh", age: -1 }

As you may have noticed we also changed the age type from a u32 to a i32 , since we had to take a negative value.

And voila! We solved the problem of receiving a JSON payload with a missing value by adding just a couple of lines of code.

Of course this scenario was over simplified and the goal was to show how you can manipulate the serialization and deserialization by using serde.rs Field Attributes.

I just wanted to quickly also show a solution to the missing field problem when deserializing a JSON payload with serde.rs and Rust without the use of any boilerplate code:

use serde::{Serialize, Deserialize};

/// The `Person` struct is the entry point into the program.
#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    age: Option,
}

fn main() {
    println!("Hello, serde.rs Field Attributes Example!");

    let serialized = "{\"name\":\"Josh\"}".to_string();
    println!("Serialized: {}", serialized);

    // Deserialize the `Person` struct from a string.
    let deserialized:Person = serde_json::from_str(&serialized).unwrap();
    println!("Deserialized: {:?}", deserialized);
}

In the above example you can see all I did was to wrap the type of my age property in an Option<T> . But doing that, when I run this code with my missing value, this is the result:

Hello, serde.rs Field Attributes Example!
Serialized: {"name":"Josh"}
Deserialized: Person { name: "Josh", age: None }

On this post I showed only one example of the Field Attributes available on serde.rs , but I highly recommend exploring all the other options, since they might help solve a lot of day to day problems and in a very simple and straight forward way.

That is all folks!

Don’t forget to follow and hit that LIKE if you like the content.