docs/build_system
Build System
There are many ways to build prism, which means the build system is a bit more complicated than usual.
Requirements
- It must work to build prism for all 6 uses-cases below.
- It must be possible to build prism without needing ruby/rake/etc. Because once prism is the single parser in TruffleRuby, JRuby or CRuby there won’t be another Ruby parser around to parse such Ruby code. Most/every Ruby implementations want to avoid depending on another Ruby during the build process as that is very brittle.
- It is desirable to compile prism with the same or very similar compiler flags for all use-cases (e.g. optimization level, warning flags, etc). Otherwise, there is the risk prism does not work correctly with those different compiler flags.
The main solution for the second point seems a Makefile, otherwise many of the usages would have to duplicate the logic to build prism.
General Design
- Templates are generated by
templates/template.rb
- The
Makefile
compiles bothlibprism.a
andlibprism.{so,dylib,dll}
from thesrc/**/*.c
andinclude/**/*.h
files - The
Rakefile
:compile
task ensures the above prerequisites are done, then callsmake
, and usesRake::ExtensionTask
to compile the C extension (using itsextconf.rb
), which useslibprism.a
This way there is minimal duplication, and each layer builds on the previous one and has its own responsibilities.
The static library exports no symbols, to avoid any conflict.
The shared library exports some symbols, and this is fine since there should only be one libprism shared library
loaded per process (i.e., at most one version of the prism gem loaded in a process, only the gem uses the shared library).
The various ways to build prism
Building from ruby/prism repository with bundle exec rake
rake
calls make
and then uses Rake::ExtensionTask
to compile the C extension (see above).
Building the prism gem by gem install/bundle install
The gem contains the pre-generated templates.
When installing the gem, extconf.rb
is used and that:
- runs
make build/libprism.a
- compiles the C extension with mkmf
When installing the gem on JRuby and TruffleRuby, no C extension is built, so instead of the last step,
there is Ruby code using FFI which uses libprism.{so,dylib,dll}
to implement the same methods as the C extension, but using serialization instead of many native calls/accesses
(JRuby does not support C extensions, serialization is faster on TruffleRuby than the C extension).
Building the prism gem from git, e.g. gem "prism", github: "ruby/prism"
The same as above, except the extconf.rb
additionally runs first:
templates/template.rb
to generate the templates
Because of course those files are not part of the git repository.
Building prism as part of CRuby
This script imports prism sources in CRuby.
The script generates the templates when importing.
prism’s Makefile
is not used at all in CRuby. Instead, CRuby’s Makefile
is used.
Building prism as part of TruffleRuby
This script imports prism sources in TruffleRuby.
The script generates the templates when importing.
Then when mx build
builds TruffleRuby and the prism
mx project inside, it runs make
.
Then the prism bindings
mx project is built, which contains the bindings
and links to libprism.a
(to avoid exporting symbols, so no conflict when installing the prism gem).
Building prism as part of JRuby
TODO, similar to TruffleRuby.
Building prism from source as a C library
All of the source files match src/**/*.c
and all of the headers match include/**/*.h
.
If you want to build prism as a shared library and link against it, you should compile with:
-fPIC -shared
- Compile as a shared library-DPRISM_EXPORT_SYMBOLS
- Export the symbols (by default nothing is exported)
Flags
make
respects the MAKEFLAGS
environment variable. As such, to speed up the build you can run:
MAKEFLAGS="-j10" bundle exec rake compile