I wonder if there's ever a point at which you could run a thread at very low "niceness" that just slowly keeps compiling more and more methods into native code as the program goes on. Surely this would be worth it for long lived server programs. You could even keep it around to periodically recompile methods if runtime information shows that some paths are hotter than others.
One reason is that the JIT needs info on the types passed to the method, which are only available after a few calls. Per the article, a+b could refer to any type of a and b. The only way to know what was used is profiling.
This technique is used by some existing VMs. .NET does this with a background thread and calls it 'tiered compilation', where methods are queued for recompilation at a higher (more expensive/thorough) optimization level in the background. It's pretty helpful.
The article targets MRI (Matzes Ruby Interpreter). It does not execute threads concurrently. Probably for the same reason that Python imposes limits on threads - to stay single threaded when calling into C libraries, most of which were written before threads were important enough to think about them
While that's true for the running program, it doesn't stop the JIT engine running a thread to compile code as the parent comment suggested. It's not running the ruby code.
Slightly longer answer: no, because Ruby as a language isn't designed to be JIT friendly. Any web request in my rails app could theoretically add new functions to any other class, in ways that would just not work in most other languages. This wreaks havoc on JIT compilers, because they constantly need to check if the assumptions they made when compiling certain bits of code still hold or if the entire world has shifted out from underneath them. This is a super beloved aspect of Ruby, but is just not JIT friendly.
While probably less common in Java, the language supports dynamic class loading, reflection, runtime bytecode generation, invokedynamic, and other features that can break static assumptions. The JVM does need to deal with all of this and does so with speculative optimization and deoptimization.
TruffleRuby applies many of these same techniques to Ruby to great effect. Fortunately, most applications stabilize at some point and a JIT compiler can do wonders. E.g., TruffleRuby has had zero-overhead metaprogramming for a decade now[1]. But, these optimizations incur trade-offs. TruffleRuby has a Ruby core library mostly written in Ruby to help optimization, but that means the interpreter takes a performance hit. The CRuby VM has most of its core library written in C because the team favors having a fast interpreter with low overhead, which is great, but means much of the core library can't be JIT compiled alongside an application.
I think the real answer why the CRuby VM lags behind the JVM is more pedestrian: there's been orders of magnitude more investment made into the JVM than any Ruby VM. Sun and then Oracle have hired world class VM developers and researchers to work on multiple JIT compilers, multiple GC implementations, fast tooling, a standard memory model, world-class concurrency library, and so on. OpenJDK sees heavy investment by many others, too (RedHat probably leading the charge). And then you have other vendors advancing the state of the art with alternative VM implementations (Azul, IBM, Amazon, etc.).
Ruby as a language has seen loads of research done on compilation and alternative VMs [2], but the CRuby VM itself hasn't seen anything close to OpenJDK's investment. That's not to say there isn't excellent work being done to improve the CRuby VM. But, production VMs are large, complex projects and not the sort of thing easily moved by a volunteer working nights and weekends. More investment by companies using Ruby would help close the gap.
While it's true that CAN happen, it doesn't mean it happens often enough to disrupt JIT compiling the code. It does mean there is an invalidation when code paths are modified.
JRuby is a thing and it runs on the JVM as more than a simple interpreter. The JVM does have to deal with this exact problem. So you could actually say Ruby can already run as fast as the JVM because it runs on it, what it cannot do is run as fast as a static language because, at the very least, it has to deal with additional checks that things haven't changed.
> Any web request in my rails app could theoretically add new functions to any other class
Java Spring applications pretty much do the same thing (reflections, proxy beans, aop, JSON/Date serializers/deserializers, ORMs, etc are all compiled and changed dynamically upon route hits).
These things are theoretically possible (and happen) in any JIT language (node, java). Only statically compiled languages seems to be immune to this.
I don’t think that answers GPs question. They asked it could be fast (a result), not if it could use a JIT (an implementation detail). I would argue it is an easy language to JIT since its semantics are clear and straightforward, just that it might not gain any speed if there aren’t any optimizations it can apply over the code run in the JIT.
If you want an example of a bad language to JIT, take C for example, where parsing and running code is hugely context dependent and adding new code can change just about anything about the existing code without anything knowing about it. And yet most C runs via a JIT: dlopen, the just in time loader. Just look at the mess that is historic ELF to attempt to deal with the problem
They asked if it could be "as fast as the JVM", which JIT is a crucial part of how the JVM achieves its performance. JIT in this context is referring to the process of a source file (.rb, .js, .c, etc), or usually bytecode, being compiled into machine code. I cannot think of an instance where a C source file is JIT compiled and dlopen(3) will not be happy if you tried to call it on a C source file.
And yet we were just talking about the JVM, which will not be happy if you tried to call it on a Java source file. Or is WASM an AOT compiler because it runs the equivalent dlopen of C code somewhat slower than the native dlopen?
Bytecode is not machine code, and I specifically said source files "or usually bytecode". A C program is compiled into machine code (and more specifically a platforms given executable format) before dlopen(3) is called on it. Not sure what point you're really trying to make.
I’m saying your definition isn’t logical since there are counter examples to those claims. For one thing, Java byte code is a machine code (https://en.m.wikipedia.org/wiki/Java_processor Jazelle). These things differ in degrees of quality (along various axes) more than in their definitions.
Something that might be useful would be a sub language that didn’t support all the dynamic features that make JIT difficult or slow. Perhaps a module could have a pragma or something to indicate the language set in use. Maybe like Racket. Simply not being able to add new methods or new member variables after initialization would help.
This. Maybe we could call a method of the JIT (it must be exposed) and tell it that we won't use some features of Ruby, globally or inside that scope. That would let it skip some checks. Of course calling that method takes time so it should be something that is called only once. It depends on how the JIT accesses Ruby code.
And if the code actually does what we declared or must not do, we accept any runtime error that it might happen.
We must trust dependencies, including Rails if it's a Rails app.
It is ruby running on a Java Virtual Machine. Imposes all downsides of the JVM (try to allocate more that 4GByte per object!) and provides JVMs concurrency model. Currently supports Ruby 3.1 (it claims to support 3.4, read the fine print if your specific feature is supported!)
Is there a way to snapshot the VM's warmed up state? So the next time when I run the application, it can jump into the warmed up state and continue from there?
This can be done. The virtual machine could dump its complete memory image into an ELF file. That way, resuming consists of simply executing the file. Emacs has done this for a while via the unexec function:
I wonder if there's ever a point at which you could run a thread at very low "niceness" that just slowly keeps compiling more and more methods into native code as the program goes on. Surely this would be worth it for long lived server programs. You could even keep it around to periodically recompile methods if runtime information shows that some paths are hotter than others.
One reason is that the JIT needs info on the types passed to the method, which are only available after a few calls. Per the article, a+b could refer to any type of a and b. The only way to know what was used is profiling.
This technique is used by some existing VMs. .NET does this with a background thread and calls it 'tiered compilation', where methods are queued for recompilation at a higher (more expensive/thorough) optimization level in the background. It's pretty helpful.
I believe most browser JS runtimes do it too.
The article targets MRI (Matzes Ruby Interpreter). It does not execute threads concurrently. Probably for the same reason that Python imposes limits on threads - to stay single threaded when calling into C libraries, most of which were written before threads were important enough to think about them
While that's true for the running program, it doesn't stop the JIT engine running a thread to compile code as the parent comment suggested. It's not running the ruby code.
Quite common across Java (including its Android cousin) and .NET implementations.
I wonder if Ruby's VM will ever become as fast as the JVM
Short answer: no.
Slightly longer answer: no, because Ruby as a language isn't designed to be JIT friendly. Any web request in my rails app could theoretically add new functions to any other class, in ways that would just not work in most other languages. This wreaks havoc on JIT compilers, because they constantly need to check if the assumptions they made when compiling certain bits of code still hold or if the entire world has shifted out from underneath them. This is a super beloved aspect of Ruby, but is just not JIT friendly.
While probably less common in Java, the language supports dynamic class loading, reflection, runtime bytecode generation, invokedynamic, and other features that can break static assumptions. The JVM does need to deal with all of this and does so with speculative optimization and deoptimization.
TruffleRuby applies many of these same techniques to Ruby to great effect. Fortunately, most applications stabilize at some point and a JIT compiler can do wonders. E.g., TruffleRuby has had zero-overhead metaprogramming for a decade now[1]. But, these optimizations incur trade-offs. TruffleRuby has a Ruby core library mostly written in Ruby to help optimization, but that means the interpreter takes a performance hit. The CRuby VM has most of its core library written in C because the team favors having a fast interpreter with low overhead, which is great, but means much of the core library can't be JIT compiled alongside an application.
I think the real answer why the CRuby VM lags behind the JVM is more pedestrian: there's been orders of magnitude more investment made into the JVM than any Ruby VM. Sun and then Oracle have hired world class VM developers and researchers to work on multiple JIT compilers, multiple GC implementations, fast tooling, a standard memory model, world-class concurrency library, and so on. OpenJDK sees heavy investment by many others, too (RedHat probably leading the charge). And then you have other vendors advancing the state of the art with alternative VM implementations (Azul, IBM, Amazon, etc.).
Ruby as a language has seen loads of research done on compilation and alternative VMs [2], but the CRuby VM itself hasn't seen anything close to OpenJDK's investment. That's not to say there isn't excellent work being done to improve the CRuby VM. But, production VMs are large, complex projects and not the sort of thing easily moved by a volunteer working nights and weekends. More investment by companies using Ruby would help close the gap.
[1] -- https://chrisseaton.com/rubytruffle/pldi15-metaprogramming/p... (FYI, the paper refers to JRuby+Truffle, which is TruffleRuby's old name).
[2] -- https://ruby-compilers.com/
While it's true that CAN happen, it doesn't mean it happens often enough to disrupt JIT compiling the code. It does mean there is an invalidation when code paths are modified.
JRuby is a thing and it runs on the JVM as more than a simple interpreter. The JVM does have to deal with this exact problem. So you could actually say Ruby can already run as fast as the JVM because it runs on it, what it cannot do is run as fast as a static language because, at the very least, it has to deal with additional checks that things haven't changed.
> Any web request in my rails app could theoretically add new functions to any other class
Java Spring applications pretty much do the same thing (reflections, proxy beans, aop, JSON/Date serializers/deserializers, ORMs, etc are all compiled and changed dynamically upon route hits).
These things are theoretically possible (and happen) in any JIT language (node, java). Only statically compiled languages seems to be immune to this.
I don’t think that answers GPs question. They asked it could be fast (a result), not if it could use a JIT (an implementation detail). I would argue it is an easy language to JIT since its semantics are clear and straightforward, just that it might not gain any speed if there aren’t any optimizations it can apply over the code run in the JIT.
If you want an example of a bad language to JIT, take C for example, where parsing and running code is hugely context dependent and adding new code can change just about anything about the existing code without anything knowing about it. And yet most C runs via a JIT: dlopen, the just in time loader. Just look at the mess that is historic ELF to attempt to deal with the problem
They asked if it could be "as fast as the JVM", which JIT is a crucial part of how the JVM achieves its performance. JIT in this context is referring to the process of a source file (.rb, .js, .c, etc), or usually bytecode, being compiled into machine code. I cannot think of an instance where a C source file is JIT compiled and dlopen(3) will not be happy if you tried to call it on a C source file.
And yet we were just talking about the JVM, which will not be happy if you tried to call it on a Java source file. Or is WASM an AOT compiler because it runs the equivalent dlopen of C code somewhat slower than the native dlopen?
Bytecode is not machine code, and I specifically said source files "or usually bytecode". A C program is compiled into machine code (and more specifically a platforms given executable format) before dlopen(3) is called on it. Not sure what point you're really trying to make.
I’m saying your definition isn’t logical since there are counter examples to those claims. For one thing, Java byte code is a machine code (https://en.m.wikipedia.org/wiki/Java_processor Jazelle). These things differ in degrees of quality (along various axes) more than in their definitions.
Hotspot was originally designed for such kind of languages, also while Java looks like C++, its semantics are closer to Objective-C.
Something that might be useful would be a sub language that didn’t support all the dynamic features that make JIT difficult or slow. Perhaps a module could have a pragma or something to indicate the language set in use. Maybe like Racket. Simply not being able to add new methods or new member variables after initialization would help.
This. Maybe we could call a method of the JIT (it must be exposed) and tell it that we won't use some features of Ruby, globally or inside that scope. That would let it skip some checks. Of course calling that method takes time so it should be something that is called only once. It depends on how the JIT accesses Ruby code.
And if the code actually does what we declared or must not do, we accept any runtime error that it might happen.
We must trust dependencies, including Rails if it's a Rails app.
With such constraints, it should also be possible to compile it into a native binary, and then it is very similar Crystal.
Are you aware of jruby?
https://www.jruby.org/
It is ruby running on a Java Virtual Machine. Imposes all downsides of the JVM (try to allocate more that 4GByte per object!) and provides JVMs concurrency model. Currently supports Ruby 3.1 (it claims to support 3.4, read the fine print if your specific feature is supported!)
Even V8 isn't as fast as JVM :) I think only LuaJIT is comparable in some ways.
Ruby can run on the JVM...
Will Ruby ever be as fast as Java? Probably not, because Ruby is dynamic (don't know types until runtime) and Java is static...
You could use Ruby on top of JVM such as TruffleRuby and JRuby.
Is there a way to snapshot the VM's warmed up state? So the next time when I run the application, it can jump into the warmed up state and continue from there?
Technically yes, this is common in JVM implementations, Android, .NET, and oldies like Smalltalk and Common Lisp.
So it could be possible to implement something similar for Ruby.
https://criu.org/
Cool, I’ll check this out and ser how applicable it is to Ruby, otherwise I’ll try to use CRaC with TruffleRuby
This can be done. The virtual machine could dump its complete memory image into an ELF file. That way, resuming consists of simply executing the file. Emacs has done this for a while via the unexec function:
https://lwn.net/Articles/673724/
The big problem is (of course) global state in C libraries, especially libc.