Repository navigation
Scaling PyO3 down with single-file Python extensions #344
Description
One nice thing about Cython is that you can have a single .pyx file, two lines in setup.py and one line in requirements.txt, and you're good. With PyO3, you need a Cargo.toml, configured a particular way (cdylib), and so it's just a bit more ceremony than Cython for simple cases.
Maturin is great for standalone libraries, but doesn't help simplify the case of adding a quick extension to an existing Python project.
To solve this, one could imagine single-file Rust extension modules, inspired by https://github.com/rust-lang/rfcs/blob/master/text/3424-cargo-script.md (and see the references to existing tools that predate the RFC, mostly for executables). You might then have:
setup.py
mypackage/
__init__.py
module.py
_extension.rs
And the setup.py is modified to point at _extension.rs, much like you would with Cython, rather than pointing at Cargo.toml as it normally does:
from setuptools import setup
from setuptools_rust import Binding, RustSingleFileExtension
setup(
name="mypackage",
version="1.0",
rust_extensions=[RustSingleFileExtension("mypackage._extension", binding=Binding.PyO3)],
packages=["mypackage"],
# rust extensions are not zip safe, just like C-extensions.
zip_safe=False,
)
The _extension.rs might look like this:
//! ```cargo
//! [dependencies]
//! pyo3 = "0.19"
//! ```
use pyo3::prelude::*;
/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
/// A Python module implemented in Rust.
#[pymodule]
fn _extension(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
Ok(())
}Once you start thinking about lock files this starts getting more complicated, they could be auto-generated as mypackage/_extension.rs.lock, and you'd need some way to update them... conceivably could outsource that to Cargo eventually if the feature becomes non-experimental and they are willing to add a new use case.
Activity
itamarst commentedon Jul 3, 2023
As an alternative to the above syntax, the config part could be specified in
setup.pyas an argument toRustSingleFileExtension(or whatever it ends up being called) rather than in the Rust file.itamarst commentedon Jul 3, 2023
Some reasons this might be a bad idea:
Cargo.toml. This can likely be solved with tooling, but that's an increase in scope.Cargo.toml-equivalent config.adamreichold commentedon Jul 4, 2023
I think if one goes automatically generating a
Cargo.lock, one could also implement this as automatically generating (or updating) aCargo.tomlfile. The canonical path ofsrc/lib.rsis only conventional IIRC and aCargo.tomlcould change this toextension.rs.itamarst commentedon Jul 5, 2023
One alternative I just discovered to the proposal here is https://github.com/mityax/rustimport
I'll try playing around with it, and report what I've learned here.
alex commentedon Jul 6, 2023
It seems to me that once the cargo-script RFC is stable, it'd be a much smaller amount of work to build this on top of that.
itamarst commentedon Jul 10, 2023
So I played around with
rustimport(https://pythonspeed.com/articles/easiest-rust-python/), and my feeling is it does quite well at simplifying prototyping. And there's maybe a use case for single-file Rust that is an extension you ship or distribute, but... it feels like prototyping is maybe the bigger use case.So from my perspective it's probably not worth spending a lot of time on adding it to
setuptools-rust, I'm happy to close this issue.itamarst commentedon Jul 17, 2023
Another alternative: https://www.maturin.rs/develop.html#import-hook