Home / Blog / DataFlex - Implementing Liquid templating by binding with Rust

27 May 2023

DataFlex - Implementing Liquid templating by binding with Rust

The Goal

We want to create dynamic emails that end users can customize as their needs change. To do this we will need a way to offer them customisations that allow for semi complex structures and rules.

Liquid is a great templating language built by Shopify which makes formatting things like emails and other dynamic content really easy. The templating includes basic control structures and decision trees which means you can make very universal templates for a lot of conditions. These templates can then be updated on the fly as they are generally parsed each call.

The way I see the approach working is:

Template creation is done in Dataflex. Any text entry field can handle this.
DataFlex will create a struct of possible values that can be used on a specific template item.
DataFlex will convert the struct to a string and pass it to a Rust Library a long with the template
The Rust library will convert the JSON string back to a JSON object
The Rust library will then parse the template with the JSON object using a Liquid templating engine
Rust will return a new string that should have the correct content
DataFLex will then handle what we use that text for
Lets go..!

Picking a Liquid library

https://github.com/cobalt-org/liquid-rust

On the face of it, this library looks perfect. It takes a JSON object, a string, and parses the string with Liquid returning a string for the result.

let template = liquid::ParserBuilder::with_stdlib()
    .build().unwrap()
    .parse("Liquid! -2").unwrap();

let mut globals = liquid::object!({
    "num": 4f64
});

let output = template.render(&globals).unwrap();
assert_eq!(output, "Liquid! 2".to_string());

Let’s create the DataFlex Rust binding library

cargo new --lib df-liquid

We will update the cargo.toml definition to make it a cdylib

[package]
name = "dataflex_liquid_bindings"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ['cdylib']

Then install the Liquid library

cargo add liquid

The initial proof of concept, passing in a c string

Including our helpers library:

use std::{os::raw::c_char, ffi::CStr};

pub fn get_c_string(c_s: *const c_char) -> String {
  let r_string: String = unsafe {
      CStr::from_ptr(c_s).to_string_lossy().into_owned()
  };

  return r_string;
}

mod tests {
  use std::os::raw::c_char;

  #[test]
  fn test_characters() {
      let result = super::get_c_string("testing😥".as_ptr() as *const c_char);
      assert_eq!(result, "testing😥");
  }
}

We can get a PoC going with

use std::{ffi::CString, os::raw::c_char};
use crate::helpers::{get_c_string};

pub mod helpers;

#[no_mangle]
pub extern fn liquidify(c_template: *const c_char) -> *mut c_char {
  let owned_template = get_c_string(c_template);
  let template = liquid::ParserBuilder::with_stdlib()
  .build().unwrap()
  .parse(owned_template.as_str()).unwrap();

  let mut globals = liquid::object!({
    "num": 4f64
  });

  let output = template.render(&globals).unwrap();
  let c_result = CString::new(output).unwrap();

  return c_result.into_raw();
}

mod tests {
  use std::os::raw::c_char;
  use crate::get_c_string;

  #[test]
  fn test_liquidify() {
    let c_template: *const c_char = "Liquid! -2\0".as_ptr() as *const c_char;

    let c_result = super::liquidify(
      c_template
    );

    let output = get_c_string(c_result);

    assert_eq!(output, "Liquid! 2")
  }
}

Running tests we succeed!

PS C:\Users\Joseph.Mullins\workspace\df-liquid> cargo test
   Compiling dataflex_liquid_bindings v0.1.0 (C:\Users\Joseph.Mullins\workspace\df-liquid)
    Finished test [unoptimized + debuginfo] target(s) in 1.21s
     Running unittests src\lib.rs (target\debug\deps\dataflex_liquid_bindings-2bcffc73adfe20ce.exe)

running 2 tests
test helpers::tests::test_characters ... ok
test tests::test_liquidify ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

This at least proves, that through a DLL, we can pass a template in as a c string, and that will return it parsed with some hardcoded JSON values.

JSON it up

Now let’s try get the JSON object coming in as a string.

We are going to generate this in DataFlex to ensure consistency.

Struct tJSONExample
    Integer iAge
    String sName
End_Struct

Procedure TestJSON
    Handle hoJSON
    tJSONExample example
    String sJSON

    Move 21 to example.iAge
    Move "Joseph Mullins" to example.sName

    Get Create (RefClass(cJsonObject)) to hoJson
    Send DataTypeToJson of hoJson example
    Set peWhiteSpace of hoJson to jpWhitespace_Spaced
    Get Stringify of hoJson to sJSON
    Send Destroy of hoJson
End_Procedure

This gets us

{ "iAge": 21, "sName": "Joseph Mullins" }

Let’s update our test with the new JSON input

  fn test_liquidify() {
    let c_template: *const c_char = "Liquid! -2 \0".as_ptr() as *const c_char;
    let c_json: *const c_char = "{ \"iAge\": 21, \"sName\": \"Joseph Mullins\" }\0".as_ptr() as *const c_char;
    let c_result = super::liquidify(
      c_template,
      c_json
    );

    let output = get_c_string(c_result);

    assert_eq!(output, "Liquid! 19 Joseph Mullins")
  }

We will add serde_json to parse the JSON

cargo add serde_json

Update the main function to handle the JSON string and parse it.

#[no_mangle]
pub extern fn liquidify(c_template: *const c_char, c_json: *const c_char) -> *mut c_char {
  let owned_template = get_c_string(c_template);
  let owned_json = get_c_string(c_json);

  let template = liquid::ParserBuilder::with_stdlib()
  .build().unwrap()
  .parse(owned_template.as_str()).unwrap();

  let v: serde_json::Value = serde_json::from_str(owned_json.as_str()).unwrap();

  let globals = liquid::to_object(&v).unwrap();

  let output = template.render(&globals).unwrap();
  let c_result = CString::new(output).unwrap();

  return c_result.into_raw();
}

Which makes our tests pass!

PS C:\Users\Joseph.Mullins\workspace\df-liquid> cargo test
    Finished test [unoptimized + debuginfo] target(s) in 0.10s
     Running unittests src\lib.rs (target\debug\deps\dataflex_liquid_bindings-9835d54a43c35fb2.exe)

running 2 tests
test helpers::tests::test_characters ... ok
test tests::test_liquidify ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

So besides error handling and all the niceties, we have a DLL that is a PoC we can start testing with DataFlex.

So let’s build it and start testing

cargo build --lib --release

Which places it in target/release/ as dataflex_liquid_bindings.dll

We copy that to our DataFlex project Programs directory for now.

Testing in DataFlex

Let’s create a new class and make this a little cleaner.

File -> New -> Class -> cLiquidTemplating

Create a new DataFlex class

We will add 2 properties, psTemplate and pvData.

psTemplate will hold the template. pvData will hold a variant of the struct.

Class cLiquidTemplating is a cObject

    Procedure Construct_Object
        Forward Send Construct_Object

        Property String psTemplate
        Property Handle phData

    End_Procedure

    Procedure End_Construct_Object
        Forward Send End_Construct_Object

    End_Procedure

    Function Parse Returns String
    End_Function

End_Class

We will add the DLL function definition to the top of that class

External_Function _R_LiquidParse "liquidify" dataflex_liquid_bindings.dll ;
    String sTemplate String sData  ;
    Returns String

Then we will flesh out the parse method.

We need to

Create a JSON object
Convert the passed struct to the JSON object
Make the JSON a string
Then call the library with those two strings, and get the result

First we will flesh out or parse method

Function Parse Returns String
    Variant vData
    Handle hoJSON
    String sTemplate sJSON sResult

    Get pvData to vData
    Get psTemplate to sTemplate

    Get Create (RefClass(cJsonObject)) to hoJson
    Send DataTypeToJson of hoJson vData
    Set peWhiteSpace of hoJson to jpWhitespace_Spaced
    Get Stringify of hoJson to sJSON
    Send Destroy of hoJson

    Move (_R_LiquidParse(sTemplate, sJSON)) to sResult
    Move (CString(sResult)) to sResult

    Function_Return sResult
End_Function

Then we will test

Struct tBodyContent
    String title
    String[] products
End_Struct

Procedure TestLiquidify
    tBodyContent bodyContent
    String sResult

    Move (InsertInArray(bodyContent.products, -1, "Apples")) to bodyContent.products
    Move (InsertInArray(bodyContent.products, -1, "Pears")) to bodyContent.products
    Move (InsertInArray(bodyContent.products, -1, "Oranges")) to bodyContent.products
    Move (InsertInArray(bodyContent.products, -1, "Grapes")) to bodyContent.products

    Object oLiquidTemplate is a cLiquidTemplating
        Set psTemplate to @"
<html>
    <head>
        <title></title>
    </head>
    <body>
        <ul></ul>
    </body>
</html>"
        Set pvData to bodyContent
    End_Object

    Get Parse of oLiquidTemplate to sResult
End_Procedure

Send TestLiquidify

The Result

And the output is…

<html>
    <head>
        <title></title>
    </head>
    <body>
        <ul>
            <li>Apples</li>
            <li>Pears</li>
            <li>Oranges</li>
            <li>Grapes</li>
        </ul>
    </body>
</html>

Now let’s do one more a little more complex

Struct tProduct
    String name
    Number price
End_Struct

Struct tBodyContent
    String title
    tProduct[] products
End_Struct

Procedure TestLiquidify
    tBodyContent bodyContent
    tProduct product
    String sResult

    Move "Apples" to product.name
    Move 120 to product.price
    Move (InsertInArray(bodyContent.products, -1, product)) to bodyContent.products

    Move "Pears" to product.name
    Move 95 to product.price
    Move (InsertInArray(bodyContent.products, -1, product)) to bodyContent.products

    Move "Oranges" to product.name
    Move 49 to product.price
    Move (InsertInArray(bodyContent.products, -1, product)) to bodyContent.products

    Move "Grapes" to product.name
    Move 180 to product.price
    Move (InsertInArray(bodyContent.products, -1, product)) to bodyContent.products

    Object oLiquidTemplate is a cLiquidTemplating
        Set psTemplate to @"
<html>
    <head>
        <title></title>
    </head>
    <body>
        <ul>
        </ul>
    </body>
</html>"
        Set pvData to bodyContent
    End_Object

    Get Parse of oLiquidTemplate to sResult
End_Procedure

Send TestLiquidify

Results in

<html>
    <head>
        <title></title>
    </head>
    <body>
        <ul>
            <li>Apples - 120</li>
            <li>Pears - 95</li>
            <li>Oranges - 49</li>
            <li>Grapes - 180</li>
        </ul>
    </body>
</html>

Success!

tags: rust - dataflex - dll - liquid