Python Bindings

Python Bindings

cellular_raza was designed such that it can be used as a numerical backend for python applications. We use pyo3 to automatically generate these bindings and install the resulting package in a local virtual environment. In the past we have created multiple projects where we used python bindings to create one concise package (cr_mech_coli, cr_trichome). It is also possible to distribute such a package on pypy.org.

cellular_raza-template-pyo3

We provide the template repository cellular_raza-template-pyo3 which makes it easy to get started with a new mixed Rust-Python project with cellular_raza.

Layout

    • cellular_raza_template_pyo3.pyi
    • init.py
        • Makefile
        • conf.py
        • index.rst
        • make.bat
        • references.bib
        • requirements.txt
        • basic.py
        • lib.rs
      • Cargo.toml
      • LICENSE
      • README.md
      • pyproject.toml
      • requirements.txt
      • ℹī¸
        Many of these files and their contents of this project should be renamed to the new project name. A full list is contained inside the README.md file.

        The source code of this project is split between the src and cellular_raza_template_pyo3 folders.

        Python Files

        The cellular_raza_template_pyo3 folder is named identically to the project and should be renamed to the new project name. It contains all python source code and is structured like a python project together with the requirements.txt and pyproject.toml files. The latter contains additional information for maturin which we use for building and publishing.

        pyproject.toml
        pyproject.toml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        
        [build-system]
        requires = ["maturin>=1.7,<2.0"]
        build-backend = "maturin"
        
        [project]
        name = "cellular_raza_template_pyo3"
        requires-python = ">=3.8"
        classifiers = [
            "Programming Language :: Rust",
            "Programming Language :: Python :: Implementation :: CPython",
            "Programming Language :: Python :: Implementation :: PyPy",
        ]
        dynamic = ["version"]
        
        [tool.maturin]
        features = ["pyo3/extension-module"]
        module-name = "cellular_raza_template_pyo3.cellular_raza_template_pyo3_rs"
        

        Within the cellular_raza_template_pyo3 folder is a *.pyi stub file which provide type information about the objects generated by pyo3. Unfortunately it is currently not possible to automatically generate these files (see github.com/PyO3/pyo3/issues/2454).

        Rust Files

        The src folder contains all code written in Rust. Together with the Cargo.toml file it acts as a regular Rust-Cargo project. It exports functions and classes inside a module from the lib.rs file.

        src/lib.rs
        135
        136
        137
        138
        139
        140
        
        #[pymodule]
        fn cellular_raza_template_pyo3_rs(m: &Bound<'_, PyModule>) -> PyResult<()> {
            m.add_class::<SimulationSettings>()?;
            m.add_function(wrap_pyfunction!(run_simulation, m)?)?;
            Ok(())
        }
        

        The name of this module should also be changed when building a new project. Functions and object need to be annotated with the #[pyfunction] and #[pyclass] derive macros respectively. See the documentation of pyo3.

        src/lib.rs
        51
        52
        53
        
        #[pyclass]
        #[derive(CellAgent, Clone, Deserialize, Serialize)]
        pub struct Agent {

        options="{"hl_lines" "51-53"}"

        src/lib.rs
        66
        67
        
        #[pyfunction]
        pub fn run_simulation(

        Docs

        The docs folder contains files for generating documentation with sphinx. It integrates well within these projects and can be configured very freely. Since we aim to provide a python package in the end, we will have to adapt the same approach in writing docstrings for our Rust-based code.

        src/lib.rs
        37
        38
        39
        
            /// Creates a new :class:`SimulationSettings` class.
            #[new]
            fn new() -> Self {

        Simulation Flow

        The overall structure of this template project is similar to that of the getting-started guide. We define an Agent struct and derive its properties from already existing building blocks.

        src/lib.rs
        50
        51
        52
        53
        54
        55
        56
        57
        58
        59
        60
        
        /// A cellular agent which is used in our simulation.
        #[pyclass]
        #[derive(CellAgent, Clone, Deserialize, Serialize)]
        pub struct Agent {
            /// Used to integrate position and velocity of our agent
            #[Mechanics]
            pub mechanics: NewtonDamped2DF32,
            /// Calculate interactions between agents
            #[Interaction]
            pub interaction: BoundLennardJonesF32,
        }

        All settings required to run a new simulation are stored in the SimulationSettings struct.

        src/lib.rs
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        
        /// Contains settings needed to specify the simulation
        #[pyclass(get_all, set_all)]
        pub struct SimulationSettings {
            /// Number of agents to initially put into the system
            pub n_agents: usize,
            /// Overall domain size
            pub domain_size: f32,
            /// Number of voxels to create subdivisions
            pub n_voxels: usize,
            /// Number of threads used
            pub n_threads: usize,
            /// Time increment used to solve the simulation
            pub dt: f32,
        }

        This information is then used to initialize all agents and the simulation domain within the run_simulation function.

        run_simulation
        src/lib.rs
         62
         63
         64
         65
         66
         67
         68
         69
         70
         71
         72
         73
         74
         75
         76
         77
         78
         79
         80
         81
         82
         83
         84
         85
         86
         87
         88
         89
         90
         91
         92
         93
         94
         95
         96
         97
         98
         99
        100
        101
        102
        103
        104
        105
        106
        107
        108
        109
        110
        111
        112
        113
        114
        115
        116
        117
        118
        119
        120
        121
        122
        123
        124
        125
        126
        127
        128
        129
        130
        131
        132
        133
        
        /// Performs a complete numerical simulation of our system.
        ///
        /// Args:
        ///     simulation_settings(SimulationSettings): The settings required to run the simulation
        #[pyfunction]
        pub fn run_simulation(
            simulation_settings: &SimulationSettings,
        ) -> Result<(), chili::SimulationError> {
            use rand::Rng;
            let mut rng = rand_chacha::ChaCha8Rng::seed_from_u64(0);
        
            // Agents setup
            let agent = Agent {
                mechanics: NewtonDamped2DF32 {
                    pos: Vector2::from([0.0, 0.0]),
                    vel: Vector2::from([0.0, 0.0]),
                    damping_constant: 1.0,
                    mass: 1.0,
                },
                interaction: BoundLennardJonesF32 {
                    epsilon: 0.01,
                    sigma: 1.0,
                    bound: 0.1,
                    cutoff: 1.0,
                },
            };
        
            let domain_size = simulation_settings.domain_size;
            let agents = (0..simulation_settings.n_agents).map(|_| {
                let mut new_agent = agent.clone();
                new_agent.set_pos(&Vector2::from([
                    rng.gen_range(0.0..domain_size),
                    rng.gen_range(0.0..domain_size),
                ]));
                new_agent
            });
        
            // Domain Setup
            let domain = CartesianCuboid2NewF32::from_boundaries_and_n_voxels(
                [0.0; 2],
                [simulation_settings.domain_size; 2],
                [simulation_settings.n_voxels; 2],
            )?;
        
            // Storage Setup
            let storage_builder = cellular_raza::prelude::StorageBuilder::new().location("out");
        
            // Time Setup
            let t0: f32 = 0.0;
            let dt = simulation_settings.dt;
            let save_points = vec![5.0, 10.0, 15.0, 20.0];
            let time_stepper = cellular_raza::prelude::time::FixedStepsize::from_partial_save_points(
                t0,
                dt,
                save_points.clone(),
            )?;
        
            let settings = chili::Settings {
                n_threads: simulation_settings.n_threads.try_into().unwrap(),
                time: time_stepper,
                storage: storage_builder,
                show_progressbar: true,
            };
        
            chili::run_simulation!(
                domain: domain,
                agents: agents,
                settings: settings,
                aspects: [Mechanics, Interaction],
            )?;
            Ok(())
        }

        This functionality is then exported as python functions.

        src/lib.rs
        135
        136
        137
        138
        139
        140
        
        #[pymodule]
        fn cellular_raza_template_pyo3_rs(m: &Bound<'_, PyModule>) -> PyResult<()> {
            m.add_class::<SimulationSettings>()?;
            m.add_function(wrap_pyfunction!(run_simulation, m)?)?;
            Ok(())
        }

        For now, the adjoining python code in ./cellular_raza_template_pyo3 is empty and only re-exports these objects and functions.

        Troubleshooting & Caveats

        pyo3 Versions

        The pyo3 version which we are using in our project and that of cellular_raza needs to match. Otherwise compilation of the project will fail.

        Generics

        Since pyo3 does not support generics, we can not expose any classes that use generics directly. If we wish to use any class with generics, we need to write wrappers around them.

        Docstrings and LSP

        Sphinx inspects the generated python object itself while most language servers rely on the inline docstrings from the python source code or *.pyi stub files. This means that the documentation of our python code which comes from Rust will show the contents of docstrings given in src/... while the syntax highlighting and completion will show the content of the stub file cellular_raza_template_pyo3/cellular_raza_template_pyo3.pyi.