How to iterate over vector of pyclass objects in rust?

Question

I am using maturin and I am trying to implement the method get_car() for my class.

But using the following code

use pyo3::prelude::*;

#[pyclass]
struct Car {
    name: String,
}

#[pyclass]
struct Garage {
    cars: Vec<Car>,
}

#[pymethods]
impl Garage {
    fn get_car(&self, name: &str) -> Option<&Car> {
        self.cars.iter().find(|car| car.name == name.to_string())
    }
}

/// A Python module implemented in Rust.
#[pymodule]
fn pyo3_iter_issue(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Car>()?;
    m.add_class::<Garage>()?;
    Ok(())
}

I get this error message

the trait bound `Car: AsPyPointer` is not satisfied
the following other types implement trait `AsPyPointer`:
  CancelledError
  IncompleteReadError
  InvalidStateError
  LimitOverrunError
  Option<T>
  PanicException
  Py<T>
  PyAny
and 107 others
required for `&Car` to implement `IntoPy<Py<PyAny>>`
1 redundant requirement hidden
required for `Option<&Car>` to implement `IntoPy<Py<PyAny>>`
required for `Option<&Car>` to implement `OkWrap<Option<&Car>>`

I am still very new to rust and I do not understand the issue here?

Answer 1

Rust does not have lifetimes, so any & references are not Python compatible.

The easiest way to fix this would be to use Car and .clone() instead:

use pyo3::prelude::*;

#[derive(Clone, Debug)]
#[pyclass]
struct Car {
    name: String,
}

#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
    cars: Vec<Car>,
}

#[pymethods]
impl Car {
    fn __str__(&self) -> String {
        format!("{:?}", self)
    }
}

#[pymethods]
impl Garage {
    fn get_car(&self, name: &str) -> Option<Car> {
        self.cars
            .iter()
            .find(|car| car.name == name.to_string())
            .cloned()
    }

    #[new]
    fn new() -> Self {
        Self {
            cars: vec![
                Car {
                    name: "Ferrari".to_string(),
                },
                Car {
                    name: "Audi".to_string(),
                },
            ],
        }
    }
}

/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Car>()?;
    m.add_class::<Garage>()?;
    Ok(())
}

#!/usr/bin/env python3

from rust_python_playground import Garage

garage = Garage()

ferrari = garage.get_car("Ferrari")
print(f"Ferrari: {ferrari}")

bugatti = garage.get_car("Bugatti")
print(f"Bugatti: {bugatti}")

Ferrari: Car { name: "Ferrari" }
Bugatti: None

Of course that would create copies of the object, which isn't always desirable.

It's a lot harder to reference something from Python code, though. Even the default get/set implementations that pyo3 can create clone the data, they do not reference.

For that, you would need a refcounter around your objects. In the standard library, this would be Rc, but Rc is managed by Rust and can therefore not be passed to Python.

The equivalent Python-managed reference counter is called Py:

use pyo3::prelude::*;

#[derive(Clone, Debug)]
#[pyclass]
struct Car {
    name: String,
    #[pyo3(get, set)]
    horsepower: u32,
}

#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
    cars: Vec<Py<Car>>,
}

#[pymethods]
impl Car {
    fn __str__(&self) -> String {
        format!("{:?}", self)
    }
}

#[pymethods]
impl Garage {
    fn get_car(&self, name: &str) -> Option<Py<Car>> {
        Python::with_gil(|py| {
            self.cars
                .iter()
                .find(|car| car.as_ref(py).borrow().name == name.to_string())
                .cloned()
        })
    }

    #[new]
    fn new() -> PyResult<Self> {
        Python::with_gil(|py| {
            Ok(Self {
                cars: vec![
                    Py::new(
                        py,
                        Car {
                            name: "Ferrari".to_string(),
                            horsepower: 430,
                        },
                    )?,
                    Py::new(
                        py,
                        Car {
                            name: "Audi".to_string(),
                            horsepower: 250,
                        },
                    )?,
                ],
            })
        })
    }

    fn __str__(&self) -> String {
        Python::with_gil(|py| {
            let mut garage_str = "[\n".to_string();
            for car in &self.cars {
                garage_str += &format!("   {:?}\n", car.as_ref(py).borrow());
            }
            garage_str += "]";
            garage_str
        })
    }
}

/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Car>()?;
    m.add_class::<Garage>()?;
    Ok(())
}

#!/usr/bin/env python3

from rust_python_playground import Garage

garage = Garage()

print(f"Garage: {garage}")
print()

ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()

print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()

print(f"Garage: {garage}")

Garage: [
   Car { name: "Ferrari", horsepower: 430 }
   Car { name: "Audi", horsepower: 250 }
]

Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None

Changing Ferrari's horsepower to >9000 ...

Garage: [
   Car { name: "Ferrari", horsepower: 9001 }
   Car { name: "Audi", horsepower: 250 }
]

This has the drawback that now every time you want to access the object, even just from within Rust code, you need a GIL lock.

The third option is to use Rc (or Arc, because of threadsafety), but don't expose it to Python directly; instead, write a CarRef wrapper that carries it. But then you might have to also use Mutex because of internal mutability, and it all gets messy pretty quickly. Although it's of course doable:

use std::sync::{Arc, Mutex};

use pyo3::prelude::*;

#[derive(Clone, Debug)]
#[pyclass]
struct Car {
    name: String,
    #[pyo3(get, set)]
    horsepower: u32,
}

#[derive(Clone, Debug)]
#[pyclass]
struct CarRef {
    car: Arc<Mutex<Car>>,
}

#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
    cars: Vec<Arc<Mutex<Car>>>,
}

#[pymethods]
impl Car {
    fn __str__(&self) -> String {
        format!("{:?}", self)
    }
}

#[pymethods]
impl CarRef {
    fn __str__(&self) -> String {
        format!("{:?}", self.car.lock().unwrap())
    }

    #[getter]
    fn get_horsepower(&self) -> PyResult<u32> {
        Ok(self.car.lock().unwrap().horsepower)
    }

    #[setter]
    fn set_horsepower(&mut self, value: u32) -> PyResult<()> {
        self.car.lock().unwrap().horsepower = value;
        Ok(())
    }
}

#[pymethods]
impl Garage {
    fn get_car(&self, name: &str) -> Option<CarRef> {
        self.cars
            .iter()
            .find(|car| car.lock().unwrap().name == name.to_string())
            .map(|car| CarRef {
                car: Arc::clone(car),
            })
    }

    #[new]
    fn new() -> PyResult<Self> {
        Ok(Self {
            cars: vec![
                Arc::new(Mutex::new(Car {
                    name: "Ferrari".to_string(),
                    horsepower: 430,
                })),
                Arc::new(Mutex::new(Car {
                    name: "Audi".to_string(),
                    horsepower: 250,
                })),
            ],
        })
    }

    fn __str__(&self) -> String {
        let mut garage_str = "[\n".to_string();
        for car in &self.cars {
            garage_str += &format!("   {:?}\n", car.lock().unwrap());
        }
        garage_str += "]";
        garage_str
    }
}

/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Car>()?;
    m.add_class::<Garage>()?;
    Ok(())
}

#!/usr/bin/env python3

from rust_python_playground import Garage

garage = Garage()

print(f"Garage: {garage}")
print()

ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()

print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()

print(f"Garage: {garage}")

Garage: [
   Car { name: "Ferrari", horsepower: 430 }
   Car { name: "Audi", horsepower: 250 }
]

Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None

Changing Ferrari's horsepower to >9000 ...

Garage: [
   Car { name: "Ferrari", horsepower: 9001 }
   Car { name: "Audi", horsepower: 250 }
]

But as you can see, it's not quite straight-forward. But now you at least avoided the GIL lock.