[erlang-questions] ets vs list

[Ulf Wiger]

On 09/13/2010 05:16 PM, Morten Krogh wrote:
>  Ulf and Robert,
> thanks for your answers. It makes sense that delete calls free in the
> malloc sense.
> 
> What I don't get now, is what data the data in ets that is not garbage
> collected,  actually is.

Hmm, not sure I understand the question, but data copied into ETS is
managed separately from the process heap data. The Erlang VM has
a system of "memory carriers" to manage the allocation of different
types of data, in order to fight memory fragmentation. ETS data, which
tends to be more long lived and of different granularity than process
heaps, is stored in separate carrier blocks. Binaries are stored in
other types of carriers, etc. But they are just regular Erlang objects.

Basically, you have a few different kinds of data inside the VM.
To name a few:
- The atom table, is never garbage-collected; atoms are never deleted
- ETS tables, whole tables are automatically removed when the owner
  process dies; individual objects remain while the table remains,
  unless explicitly deleted
- Large binaries, reside on a shared heap; they are reference-counted
- Process heaps, are managed with a stop-and-copy garbage collector;
  when a process dies, the whole process heap is freed, informing
  monitoring and linked processes, as well as decrementing binary
  reference counters and deleting any ETS tables owned by the process.

> Does the ets process have data beyond the table itself?
> Are we talking about intermediate values used in the match function etc?


Ah, the select function actually executes in a special 'pseudo process'
inside the Erlang VM. It has its own little stack machine, called the
PAM ("PAM" stands for "Patrik's Abstract Machine", I think, where
"Patrik" stands for Patrik Nyblom).

erts/emulator/beam/erl_db_util.c

So garbage resulting from a select operation accumulates on the
pseudo process heap and is subject to the same kind of GC as
any regular process (I assume).

There's a _lot_ to memory management in the Erlang VM. I don't
claim to understand more than a tiny part of it.

BR,
Ulf W