Inside BEAM, the Erlang Virtual Machine

OTP R11B-5 includes more than one million lines of Erlang. The kernel and standard library are about a third of these lines. The emulator, BEAM, is around 200,000 lines of C.

BEAM's C main functions are platform-specific and live in erts/emulator/sys/unix, win32, and so on. (Windows actually has two mains: a simple one similar to the Unix entrypoints in erl_main.c, and a more complex one in erl_main_sae.c which is used in bootstrapping the toolchain.) The conventional mains immediately call erl_start which processes command line arguments, copies argv into a list of strings, creates a process and runs otp_ring0 with the argument list. otp_ring0 looks up and runs init:boot/1. There is more initialization after this, but from otp_ring0 on, apart from some built-in functions, Erlang is implemented in Erlang. 

In practice, the Erlang compiler (another 30,000 lines of Erlang) compiles Erlang to BEAM bytecodes. When erl_start runs otp_ring0 and so on, it is loading BEAM binaries generated by the Erlang compiler earlier. The BEAM instruction set is register-based and has 130 instructions. These include instructions for arithmetic, comparsions and boolean logic; manipulating strings, tuples and lists; stack and heap allocation and freeing; type tests for Erlang primitives (numbers, lists, process IDs, references, and so on);  jumps, structured exception handling, and calls and returns; sending messages and reading from the process' mailbox; waiting and timeouts; and so on. The instructions are listed in lib/compiler/src/beam_opcodes.erl. In BEAM, erts/emulator/beam/ops.tab contains rewrite rules that map typed BEAM instructions into emulator internals. A Perl script (beam_makeops) translates this file into C code (mostly macros.)

Some Erlang built-in function (BIFs) calls are translated into an extended instruction set not generated by the compiler. For example, the BIF erlang:self/3 is rewritten as a single 'self' instruction. Other BIF linkage is via a set of instructions for calling BIFs of varying arities. erts/emulator/beam/bif.tab maps BIF labels into C externs. For example, calls to the BIF erlang:spawn/3 boil down to a call to spawn_3 (found in erts/emulator/bif.c), which in turn thunks to erl_create_process.

Incidentally, processes are not related to threads or operating system processes. Instead, BEAM maintains queues of processes that are scheduled to run. There is one queue per priority level. Schedulers pick processes from the queues and run them. Processes maintain their registers and a pointer to the next instruction to execute.

The garbage collector, in erts/emulator/beam/erl_gc.c, is a mark-and-sweep copying collector. The most interesting thing about the BEAM garbage collector is that the garbage collected heaps are per-process. BEAM can do this because Erlang processes communicate explicitly via messages, and not via mutating shared memory. The CLR, in contrast, divides heaps into generations protected by write barriers and does incremental collection to avoid having to garbage collect all of a program's heap at the same time. This extra complexity is part of the price of using a shared heap to communicate between threads in the CLR.