Rendered at 22:43:33 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
O3marchnative 2 hours ago [-]
The author mentions Rust's portable SIMD library [0]. The only issue with portable SIMD is it's only available on nightly. I used it in my FFT crate, but we had to switch to the fearless_simd crate in order to get a portable SIMD solution that works on stable [1].
My heard hurts - i was stupid enough to think that SIMD was a CPU only thing - I don't understand why it would be ported to GPU - huge kudos to managing to surprise me
chlorion 2 hours ago [-]
GPUs work on vectors and matrices very often, that's what they are good at, so it makes a lot of sense that they can operate with SIMD I think!
monocasa 1 hours ago [-]
GPU "cores" are basically what a CPU would call SIMD lanes. So a GPU with 1024 'CUDA cores' might be structured as 16 relatively independent pieces that a CPU might call a core, each with a 64 wide SIMD unit.
mathisfun123 39 minutes ago [-]
32 wide - only AMD has a 64 wide mode
ismailmaj 2 hours ago [-]
There is something very SIMD-coded in GPU programming which is coalesced stores/loads, if a warp (32 threads) handles contiguous memory, it will create ~4 transactions instead of 32.
hingler36 3 hours ago [-]
Welcome to the lucky 10,000! SIMD is actually a pretty integral part of how GPUs are able to work efficiently, it's part of why there's such a strong focus on branchless programming in the field.
nynx 2 hours ago [-]
Do you have examples of complex algorithms running on the gpu with rust with competative performance? Radix sort might be a good one to start with
LegNeato 3 hours ago [-]
Author here, AMA.
lbhdc 3 hours ago [-]
What is vectorware's business model? Are you planning to sell support/consulting to companies using your stack? Or are you looking to sell licenses to your tool? Or something else?
LegNeato 2 hours ago [-]
The tentative plan is to open source all the compiler and `std` bits with our products built on top (compilers are not good businesses). More about our products coming in the next couple of months!
lbhdc 2 hours ago [-]
Looking forward to reading more about it. Good luck on the launch :)
shay_ker 10 minutes ago [-]
Hm is the intent to one day replace the CPU?
LegNeato 7 minutes ago [-]
The goal is to use similar abstractions and code across both the CPU and GPU where it makes sense.
jcranmer 3 hours ago [-]
The post is kind of vague on the IR you're targeting. Can you give some examples of what the SIMD-ized IR looks like, and how it maps to the target PTX?
the__alchemist 3 hours ago [-]
I'm confused too. How does this fit between these approaches for paraellization:
- CUDA kernels and Tiles (e.g. Cudarc, cuda-oxide, rust-gpu etc) - SIMD on the GPU. (E.g. as in the title...)
- CPU SIMD using avx or SSE instructions (And probably thin wrappers for vectors so you can have sane syntax). Or the maybe-upcoming core simd which should abstract over architecture-specific instructions. Magic floats etc which do 4-16 computations at once, but are a bit clumsy to work with
- Rayon thread pools - arbitrary parallel computations, including SIMD, one per CPU core.
It looks like from the code samples like maybe a cleaner syntax for writing code on the GPU than CUDA kernels? E.g. without mucking with serialization, host and device by abstracting over it? And inspired by core::simd. (Good choice if so, in the interest of standardizing on syntax; I did this for my x86 SIMD vector/quaternion lib as well)
LegNeato 2 hours ago [-]
Didn't want to go into crazy detail in the post.
Each family of operations is a trait parameterized by the operation itself:
pub trait EvaluateReduction<Operation, T>: LaneEvaluator {
/// Reduce one distributed definition to an ordinary uniform scalar.
fn evaluate_reduction(&self, value: LaneValue<Self, role::Distributed, T>) -> T;
}
Call sites name the operation:
let one = evaluator.splat::<Splat, _>(1_u32);
let two = evaluator.splat::<Splat, _>(2_u32);
let three = evaluator.binary::<Add, _>(one, two);
let total = evaluator.reduce::<Sum, u32>(three); // a uniform u32
let running = <Executor as EvaluateScan<Scan<Sum, Exclusive>, u32>>::scan(&evaluator, three);
Operations like Sum, Max, ReduceXor, Inclusive, and Exclusive are all distinct types.
As mentioned in the post, execution shape is typed too. A static shuffle takes its control as a type-level constant, and the shuffle mode constrains which controls are expressible:
// Shift down one lane, keeping our own value where the source is inactive.
let down = <Executor as EvaluateShuffle<Shuffle<Down>, DownOrSelf<1>, u32>>::shuffle(&ev, v);
// Broadcast from lane zero.
let bcast = <Executor as EvaluateShuffle<Shuffle<Broadcast>, WarpLane<0>, u32>>::shuffle(&ev, down);
// Butterfly exchange with the neighbor one bit away.
let bfly = <Executor as EvaluateShuffle<Shuffle<Xor>, Butterfly<1>, u32>>::shuffle(&ev, bcast);
For an example of errors caught, a warp-scoped executor for a device-scoped barrier is a compile error:
<ScopedWarpExecutor<'_, WarpUniform> as EvaluateBarrier<Barrier<Device>>>::barrier(evaluator)
// error[E0277]: the trait bound `Device: NvptxBarrierScope` is not satisfied
// help: the trait `NvptxBarrierScope` is implemented for `Warp`
Strip mining is typed on the amount of work and the lane capacity, and it hands back one chunk at a time along with the predicate saying which lanes live in that chunk:
// Six work items across four active lanes: two chunks, based at 0 and 4.
<Executor as EvaluateStripMine<StripMine, (WorkItems, ActiveLanes<StripMined<4>>), i32>>::
for_each_strip_mined(
&evaluator,
(WorkItems::new(6)?, ActiveLanes::new(4)?),
|index, active| {
// ...
},
);
Hopefully that gives the flavor of it.
lbhdc 3 hours ago [-]
This is really cool! It sounds like y'all have a compiler fork that you are using to make this work. I wanna tinker with this, is your compiler available?
LegNeato 2 hours ago [-]
It is not currently available but we intend to make it available after we launch our products.
Eridrus 2 hours ago [-]
Given the massive demand for GPUs for LLMs, what sorts of work do you expect to economically benefit from utilizing GPUs more?
LegNeato 2 hours ago [-]
Part of our thesis is that decent GPUs are in every shipping device and most software doesn't use them and should.
Eridrus 2 hours ago [-]
I guess you're looking at consumer hardware then since servers have exactly what you pay for.
Can you say more about the application space you're targeting?
PoignardAzur 1 hours ago [-]
Any thoughts about SIMD-related crates?
adityazero 11 minutes ago [-]
[dead]
guess__who 3 hours ago [-]
[flagged]
nperez19 12 minutes ago [-]
Love the pendantic mode setting on the website
minraws 7 minutes ago [-]
I don't get what's the value of it not being enabled by default what does the toggle get us, really? Maybe I don't understand web design and it makes it harder to read for some, I am dyslexic and never had any issues.
7 minutes ago [-]
efnx 3 hours ago [-]
Congrats to the Rust-GPU folks! Nice to see the good work flowing.
queisoy 3 hours ago [-]
[flagged]
rust-lang 3 hours ago [-]
Good job!
donald-trump 1 hours ago [-]
[flagged]
the__alchemist 3 hours ago [-]
Hey - this is probably off-topic/meta, but what is going on with the comments here? Is it bots?
dev_l1x_be 3 hours ago [-]
No idea, but it seems HN needs POW challenges.
lukan 2 hours ago [-]
Could also just be trolls attracted by the Rust topic.
dev_l1x_be 54 minutes ago [-]
I bet some kid is bored out of his mind and wrote a bot.
donald-trump 1 hours ago [-]
[flagged]
3 hours ago [-]
lx-user 3 hours ago [-]
[flagged]
3 hours ago [-]
guess__who 3 hours ago [-]
[flagged]
LegNeato 3 hours ago [-]
We never mention anything about superiority nor compare with other languages or programming models. This post is about making existing Rust CPU code work on the GPU.
guess__who 3 hours ago [-]
I can read between the lines and the sentiment is just to 'show them the right way'.
HN people are too nice to acknowledge this. But deep down you know this is true.
fluffybucktsnek 1 minutes ago [-]
Those aren't lines you are reading. Those are your hallucinations.
At worst, the post reads like a propaganda for VectorWare, but, overall, it reads more like their insights on the matter.
kooi 3 hours ago [-]
Useless blabbering.
If you have real feedback, great, but it's useless to rip on the hard work of others without it.
fire_wheel 3 hours ago [-]
I am rewriting some components in rust - I dont like such sentiment. It negatively impacts the community.
Rust is better in so many ways.
throwaway894345 3 hours ago [-]
I'm not a Rust user apart from an occasional toy program here and there, but you seem really triggered about a language that other people use. What's the issue?
the__alchemist 3 hours ago [-]
I care because it means I can use this in a Rust program without a FFI barrier. Regrettably, we have built computing infrastructure as a society with many barriers; programming language is one.
extrem-RAM-shor 3 hours ago [-]
Rust is the best. No other language is fun enough to program.
[0] https://doc.rust-lang.org/std/simd/index.html
[1] https://github.com/linebender/fearless_simd
They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?
SIMD seems to me, to be very platform specific. Maybe there are times one SIMD unit is not anothers' SIMD unit?
There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width.
But the design and documentation of portable_simd makes the fixed size syntactically easy/the default and the width agnostic code harder.
Each family of operations is a trait parameterized by the operation itself:
Call sites name the operation: Operations like Sum, Max, ReduceXor, Inclusive, and Exclusive are all distinct types.As mentioned in the post, execution shape is typed too. A static shuffle takes its control as a type-level constant, and the shuffle mode constrains which controls are expressible:
For an example of errors caught, a warp-scoped executor for a device-scoped barrier is a compile error: Strip mining is typed on the amount of work and the lane capacity, and it hands back one chunk at a time along with the predicate saying which lanes live in that chunk: Hopefully that gives the flavor of it.Can you say more about the application space you're targeting?
HN people are too nice to acknowledge this. But deep down you know this is true.
At worst, the post reads like a propaganda for VectorWare, but, overall, it reads more like their insights on the matter.
If you have real feedback, great, but it's useless to rip on the hard work of others without it.
Rust is better in so many ways.