It doesn't have all the bells and whistles - so no support for annotations or arrays, but for ~2h of work, it shows what is possible without resorting to bytecode generation.
This is interesting. Java desperately needs an array of struct for type safe sugar over high performance arenas, but the areas you’d turn to this would be in a zero allocation effort where the cost of the this library’s off-heap and the object allocation in the getters and setters etc largely negate the advantages for a lot of use cases.
Yup. Totally agree. Java does needs an array of structs. Hopefully value classes will help out through flattened array. But in future, one can use value records with this library with probable zero cost allocation. But the library doesn't use any reflection calls for get and set hence high performance as a result, and using records helps a lot with escape analysis. Planning to do some serious benchmarks soon. Some preliminary tests shows it's similar to c code (example code in test package). Performance suffers if record fields are arrays due to heap allocation of arrays.
The thing I coded where I felt the weight of the GC the most was a chess engine in Java that needed transposition tables. Like using regular HashMap(s) or anything similar it was too slow to really speed up the engine. If my son had stayed interested in chess I would have coded up an off-heap transposition tables but he switched to guitar which changed my side projects.
I'm glad they saw the light. Last time I used Java was in high school when it was version 7, when it was pure OOP. Didn't even have lambdas. After I learned other languages, I didn't want to use Java again, seemed like a lot of boilerplate for something that didn't even give good performance.
I've looked, it's better now, hence my comment that they saw the light. They got lambdas, records, and auto types. More importantly they got virtual threads, but that's unrelated.
Why does the array need to be immutable? Isn’t it enough to allocate the pessimistic max size of the record times the size of the array? In go slices work quite nicely to deal with “immutable” arrays and still be able to work on views on those arrays while keeping the same memory backing.
The first is that value types themselves are immutable. This affects code generation and optimization. If you were to modify the value with unmanaged code then you may not observe the modification properly from managed code. Maybe this restriction will get relaxed, but I don't see that on any roadmap any time soon.
The second problem is that value types are still nullable. The flattened array is not going to be identical to a Go slice or a C# Span etc. because it has to track the nullness of each element. It seems they don't want to nail down the exact storage format for that yet, possibly to change it in the future, and possibly because they want to add language-level control over nullability eventually too.
I find it weird that the people steering Java have been seemingly willing to sit out the use case high performance computation while it has so dominated the computing landscape. They are just patiently incrementally iterating on all these JEP's that would support dramatically improved capabilities and make Java a very attractive platform for ML - but they keep fretting over minor interface adjustments, cycle after cycle. I get there is a philosophy of keeping the language stable and well designed, but this is really taking it to an extreme in the face of missing an entire segment of computing.
It's a long roadmap, but this is their ultimate objective. Once java has value classes, future carrier classes and member patterns, that's when we shall see some very huge interest for java in ML. Also, they plan to introduce typeclasses in which it will be convenient to introduce operator overloading, and collection/array literals etc syntax. The idea is to unify the types in java, and then enable much stronger semantics to ensure data oriented programming becomes ergonomic in java.
FWIW virtual threads are great this time around. That’s something whose time has come generally.
But yeah for me Java took a big leap forward when it got lambda but there hasn’t been another big leap since. Still so many obvious features and sugar to add to file off the sharp edges and remnants of enterpriseness.
I did something similar a few years back, with a slightly different approach to declaration, using interfaces to denote the layout of the struct. Mutation was opt-in by exposing setters using the (of the time) standard JavaBeans layout and an annotation processor took care of the codegen of an implementing class, which could be used where you wanted an on-heap box of an off-heap structure.
One benefit of this approach was that by using the interface as the type you could fairly easily support a flyweight pattern, reducing GC pressure when working with large off-heap collections. The parallels between stateless interfaces and offheap structs was also quite pleasing.
I'd love to see a similar effort using more modern techniques than Unsafe et al.
I had tested it and it's quite fast. Actually, you don't need to generate any bytecode on the fly. Problem is when you deal with array as fields, implementation becomes difficult. You can revisit if interested to come back to such an implementation one day.
What is the positioning for this and how does it work? A comparison to SBE might be nice.
I understand the issue about using Layout and MemorySegment being verbose but the reason I'm using those things it to develop high performance software that uses off-help memory and bypasses object allocation.
What does "map Java record types onto native memory" actually mean? Did you somehow turn a Java record into a flyweight or is `Point point = points.get(0);` just instantiating a record instance using data read from off-help memory? If it's a dynamic mapping library using reflection, that's cool but doesn't it kill the performance goals for most Java off heap usage?
Is this more of a off-heap to heap bridge for pulling data into the normal Java space when performance isn't critical?
I use c-struct layout. I should be more explicit in the readme. I use classfile api to generate bytecode during initialisation of the Mem<T> and bytecode stored in cache in case if initialised again somewhere based on the same record type (I don't cache for records that are declared locally in a method). The class created from implementing Mem is a hidden class. So, basically, given a record, one can be able to analyse the layout based on record state description, and then for that Mem implementation (hidden class) we generate static final field varhandles + layout, segment is an instance field, and then generate bytecode the get and set to avoid reflection (actually, this is where most headache is in implementation). Go to the test package and see simple code for some adhoc rudimentary java (and native) files for benchmarks. Planning to test JMH benchmarks soon.
I'm actually planning to resurrect a dead raytracer project (https://github.com/mambastudio/MambaTracer) that has a GPU backend. And possible develop a future 2D API (a lot of work I know). There is an interesting world in multipurpose programming of GPUs, and it took me a while to realise prefix sum is the “hello world” in that area. As a matter of fact, most GPU code without efficient GPU prefix sum, don’t perform well for specific algorithms that might require some scan data, and that’s where CUDA dominates, which has an amazing implementation, and not in other areas such as OpenCL. The raytracer I was making, I encountered a bug that took me I think a month to deduce after some frustration, based on a discovery of my rudimentary struct implementation (through classes and annotation – which I totally agree with you they are a bit annoying) had an misalignment layout which made me rethink, something is wrong with my model approach. So, I needed to solve data and layout representation between java and native code, to be simple, concise, with little headache. If you notice the TypedMemory, the idea is to ensure a user can implement their own libraries with data models without depending on it, and thus becomes a simple plug. Unfortunately, @size will make your code have it as a dependency. Hopefully one day we have multifields as stack allocated arrays (https://web.archive.org/web/20251101203905/https://cr.openjd...) , hence maybe have value `record(char c, float[3] array) {}`
Ha, I wrote something like this ages ago when I had to interact with a Java application and really wanted/needed Python utilities. I created my own concurrent, type-safe DDL to shepherd that data.
I have not used SBE but looking at it, my understanding is that it starts from an explicit schema, typically XML, and generates encoder/decoder flyweights over a binary buffer. That gives much more control to the user in terms of field order (very important), sizes. Here, TypedMemory takes a different starting point in which the Java record shape is the schema, and the library derives the FFM MemoryLayout/accessors from that. I think the difference is schema/codegen/protocol orientation vs Java-type/FFM/in memorylayout orientation.
Inspired by this, I wrote a quick prototype using MethodHandle combinators: https://gist.github.com/DasBrain/19804df69c78cee257dd0294b00...
It doesn't have all the bells and whistles - so no support for annotations or arrays, but for ~2h of work, it shows what is possible without resorting to bytecode generation.
Great stuff!!!
This is interesting. Java desperately needs an array of struct for type safe sugar over high performance arenas, but the areas you’d turn to this would be in a zero allocation effort where the cost of the this library’s off-heap and the object allocation in the getters and setters etc largely negate the advantages for a lot of use cases.
Yup. Totally agree. Java does needs an array of structs. Hopefully value classes will help out through flattened array. But in future, one can use value records with this library with probable zero cost allocation. But the library doesn't use any reflection calls for get and set hence high performance as a result, and using records helps a lot with escape analysis. Planning to do some serious benchmarks soon. Some preliminary tests shows it's similar to c code (example code in test package). Performance suffers if record fields are arrays due to heap allocation of arrays.
The thing I coded where I felt the weight of the GC the most was a chess engine in Java that needed transposition tables. Like using regular HashMap(s) or anything similar it was too slow to really speed up the engine. If my son had stayed interested in chess I would have coded up an off-heap transposition tables but he switched to guitar which changed my side projects.
Hope you come back. Would be cool to venture in this new data oriented programming phase java has invested a lot in.
I'm glad they saw the light. Last time I used Java was in high school when it was version 7, when it was pure OOP. Didn't even have lambdas. After I learned other languages, I didn't want to use Java again, seemed like a lot of boilerplate for something that didn't even give good performance.
I use Java all the time for ordinary programming at work, I think it is great, but I'm not in a hurry to mess with stuff off-heap.
Maybe you should check again if the last time you used it was Java 7.
I've looked, it's better now, hence my comment that they saw the light. They got lambdas, records, and auto types. More importantly they got virtual threads, but that's unrelated.
Java has always given good performance for most of my use cases. Like backend servers where startup time is mostly irrelevant.
I doubt value classes will be helpful here because the array would have to be immutable. Context: https://openjdk.org/jeps/401
Why does the array need to be immutable? Isn’t it enough to allocate the pessimistic max size of the record times the size of the array? In go slices work quite nicely to deal with “immutable” arrays and still be able to work on views on those arrays while keeping the same memory backing.
There's two problems as I see it.
The first is that value types themselves are immutable. This affects code generation and optimization. If you were to modify the value with unmanaged code then you may not observe the modification properly from managed code. Maybe this restriction will get relaxed, but I don't see that on any roadmap any time soon.
The second problem is that value types are still nullable. The flattened array is not going to be identical to a Go slice or a C# Span etc. because it has to track the nullness of each element. It seems they don't want to nail down the exact storage format for that yet, possibly to change it in the future, and possibly because they want to add language-level control over nullability eventually too.
Yeaaah. You might be right. Hopefully we have this one day https://openjdk.org/jeps/8261007
I find it weird that the people steering Java have been seemingly willing to sit out the use case high performance computation while it has so dominated the computing landscape. They are just patiently incrementally iterating on all these JEP's that would support dramatically improved capabilities and make Java a very attractive platform for ML - but they keep fretting over minor interface adjustments, cycle after cycle. I get there is a philosophy of keeping the language stable and well designed, but this is really taking it to an extreme in the face of missing an entire segment of computing.
It's a long roadmap, but this is their ultimate objective. Once java has value classes, future carrier classes and member patterns, that's when we shall see some very huge interest for java in ML. Also, they plan to introduce typeclasses in which it will be convenient to introduce operator overloading, and collection/array literals etc syntax. The idea is to unify the types in java, and then enable much stronger semantics to ensure data oriented programming becomes ergonomic in java.
Valuetypes has been on the roadmap forever, meanwhile they put in the effort and bought out greenthreads out of retirement as virtual threads.
Even if they started properly on valuetypes today, it's probably still a 10 year slog to get the ecosystem to follow.
FWIW virtual threads are great this time around. That’s something whose time has come generally.
But yeah for me Java took a big leap forward when it got lambda but there hasn’t been another big leap since. Still so many obvious features and sugar to add to file off the sharp edges and remnants of enterpriseness.
Value types are a work in progress. It won't be (another) 10 years. Give it a few more releases.
Yup. They started transferring to the main line, but will require many tests to know if they have any issues. https://github.com/openjdk/jdk/pull/31120
I did something similar a few years back, with a slightly different approach to declaration, using interfaces to denote the layout of the struct. Mutation was opt-in by exposing setters using the (of the time) standard JavaBeans layout and an annotation processor took care of the codegen of an implementing class, which could be used where you wanted an on-heap box of an off-heap structure.
One benefit of this approach was that by using the interface as the type you could fairly easily support a flyweight pattern, reducing GC pressure when working with large off-heap collections. The parallels between stateless interfaces and offheap structs was also quite pleasing.
I'd love to see a similar effort using more modern techniques than Unsafe et al.
Interesting approach. I think Project Babylon did the same thing https://github.com/openjdk/babylon/blob/code-reflection/hat/...
I had tested it and it's quite fast. Actually, you don't need to generate any bytecode on the fly. Problem is when you deal with array as fields, implementation becomes difficult. You can revisit if interested to come back to such an implementation one day.
What is the positioning for this and how does it work? A comparison to SBE might be nice.
I understand the issue about using Layout and MemorySegment being verbose but the reason I'm using those things it to develop high performance software that uses off-help memory and bypasses object allocation.
What does "map Java record types onto native memory" actually mean? Did you somehow turn a Java record into a flyweight or is `Point point = points.get(0);` just instantiating a record instance using data read from off-help memory? If it's a dynamic mapping library using reflection, that's cool but doesn't it kill the performance goals for most Java off heap usage?
Is this more of a off-heap to heap bridge for pulling data into the normal Java space when performance isn't critical?
I use c-struct layout. I should be more explicit in the readme. I use classfile api to generate bytecode during initialisation of the Mem<T> and bytecode stored in cache in case if initialised again somewhere based on the same record type (I don't cache for records that are declared locally in a method). The class created from implementing Mem is a hidden class. So, basically, given a record, one can be able to analyse the layout based on record state description, and then for that Mem implementation (hidden class) we generate static final field varhandles + layout, segment is an instance field, and then generate bytecode the get and set to avoid reflection (actually, this is where most headache is in implementation). Go to the test package and see simple code for some adhoc rudimentary java (and native) files for benchmarks. Planning to test JMH benchmarks soon.
I don't usually like annotations in Java, but I'm rather liking your use of @size here.
Are you working on anything in particular that drove you to build this? Looks great!I'm actually planning to resurrect a dead raytracer project (https://github.com/mambastudio/MambaTracer) that has a GPU backend. And possible develop a future 2D API (a lot of work I know). There is an interesting world in multipurpose programming of GPUs, and it took me a while to realise prefix sum is the “hello world” in that area. As a matter of fact, most GPU code without efficient GPU prefix sum, don’t perform well for specific algorithms that might require some scan data, and that’s where CUDA dominates, which has an amazing implementation, and not in other areas such as OpenCL. The raytracer I was making, I encountered a bug that took me I think a month to deduce after some frustration, based on a discovery of my rudimentary struct implementation (through classes and annotation – which I totally agree with you they are a bit annoying) had an misalignment layout which made me rethink, something is wrong with my model approach. So, I needed to solve data and layout representation between java and native code, to be simple, concise, with little headache. If you notice the TypedMemory, the idea is to ensure a user can implement their own libraries with data models without depending on it, and thus becomes a simple plug. Unfortunately, @size will make your code have it as a dependency. Hopefully one day we have multifields as stack allocated arrays (https://web.archive.org/web/20251101203905/https://cr.openjd...) , hence maybe have value `record(char c, float[3] array) {}`
Ha, I wrote something like this ages ago when I had to interact with a Java application and really wanted/needed Python utilities. I created my own concurrent, type-safe DDL to shepherd that data.
At first glance it reminds me of C#'s Span<T>.
Hahaha... inspired by it actually.
This is very similar to SBE encoder/decoder flyweights over raw memory? What are the differences?
I have not used SBE but looking at it, my understanding is that it starts from an explicit schema, typically XML, and generates encoder/decoder flyweights over a binary buffer. That gives much more control to the user in terms of field order (very important), sizes. Here, TypedMemory takes a different starting point in which the Java record shape is the schema, and the library derives the FFM MemoryLayout/accessors from that. I think the difference is schema/codegen/protocol orientation vs Java-type/FFM/in memorylayout orientation.
Why not use graal?
Wouldn’t apache arrow serve the same purpose?
Nice. Very clean api.
Thanks. Main goal. Unions is where I decided to pause - no simple and ergonomic way to do it at the moment.