Posted by kobaltGIT 7 hours ago
You don’t state StatCraft is clean room, but all Artcraft software is.
Talk to a lawyer to check if you’re not wasting your time.
> It's not about memory safety per se. It's about not losing an hour of analysis because a malformed .sav file or a zero-variance sample triggered a segfault or silent wrong output. SPSS crashes on bad data.
It's not rust specific, but any language which allows you to work on a higher level and with better typing helps close that gap. For example for a long time Linux kernel had an issue with marking user/kernel addresses which opened it up to bugs and security issues. In C those are just numbers. In Rust this is moot and correct by default because user addresses are explicitly UserPtr.
So yes, sometimes a better language allows you make some unintended things explicitly not compile. It won't solve the issue completely, but the scope of the mismatch gets much smaller.
You can take a lot more "smart" shortcuts in Nim, that are hard to reason about, and probably even harder when an automated system writes them.
We have R/tidyverse which does 90% of everything and has an amazing community and procedures eg. around documentation. For everything else you have excel or python. So I don’t see the use case.
I’ve long since switched to python & other normal programming for any stats needs. And I only ever scraped the surface of SPSS/PSPP anyways. But, I’d be interested to hear what PSPP doesn’t cover.
Good luck on your project.
GNU Octave is like Matlab. Matplotlib is like Matlab but doesn't support the same syntax.
Will it have a DataFrame API?
Before R data.frame and Pandas DataFrame and substrait.dataframe, there were labeled arrays in NumPy.
Pint uses numpy ufuncs to type numbers with physical units.
Here are the notes I had for pgrust a few months ago: https://news.ycombinator.com/item?id=49112449 ; Polars (already has tested SIMD and GPU-optimized routines), substrait interface, and file formats: LanceDB, Vortex, Arrow recordbatch
Here are my notes for the ArtCraft folks about methods for Rust ports (and whether it's necessary to also port Blender to Rust with an O3/Maturin Python API) https://github.com/storytold/artcraft/issues/2489 :
> port the tests [and check figures] first; use branch coverage metrics when testing; use cargo-llvm-cov and rust-nightly to test; make (.sh,) scripts that the .github/workflows/ call so that local builds are easy; run local builds using nektos/act; use rstest to keep tests and test fixtures maintainable; use cargo-insta to snapshot; use cargo-fuzz, and do coverage-guided fuzzing; use TLA+, Lean or similar formal methods and tools to model the critical parts of the app at least
In the ArtCraft issues, there are so many great ideas that any or many or all firms could benefit from.
You even let your LLM write the README.
Look, we nearly all use LLMs these days, but the phrasing of "I am... ...alone" is particularly weird when it's obviously not just you, it's you with LLMs.
Maybe title it as "A rewrite of IBM SPSS Statistics in Rust with no unsafe" - the 100% safe claim is something an LLM makes, it's not something a reasonable human would say, as it implies the Rust is totally safe, whereas you're trying to communicate that your Rust code uses no unsafe blocks, and these are two different things.