[Erlang Systems]

4 Transactions and other access contexts

This chapter describes the Mnesia transaction system and the transaction properties which make Mnesia a fault tolerant, distributed database management system.

Also covered in this chapter are the locking functions, including table locks and sticky locks, as well as alternative functions which bypass the transaction system in favor of improved speed and reduced overheads. These functions are called "dirty operations". We also describe the usage of nested transactions. This chapter contains the following sections:

4.1 Transaction Properties

Transactions are an important tool when designing fault tolerant, distributed systems. A Mnesia transaction is a mechanism by which a series of database operations can be executed as one functional block. The functional block which is run as a transaction is called a Functional Object (Fun), and this code can read, write, or delete Mnesia records. The Fun is evaluated as a transaction which either commits, or aborts. If a transaction succeeds in executing Fun it will replicate the action on all nodes involved, or abort if an error occurs.

The following example shows a transaction which raises the salary of certain employee numbers.

raise(Eno, Raise) ->
    F = fun() ->
                [E] = mnesia:read({employee, Eno}),
                Salary = E#employee.salary + Raise,
                New = E#employee{salary = Salary},
                mnesia:write(New)
        end,
    mnesia:transaction(F).

The transaction raise(Eno, Raise) - > contains a Fun made up of four lines of code. This Fun is called by the statement mnesia:transaction(F) and returns a value.

The Mnesia transaction system facilitates the construction of reliable, distributed systems by providing the following important properties:

4.1.1 Atomicity

Atomicity means that database changes which are executed by a transaction take effect on all nodes involved, or on none of the nodes. In other words, the transaction either succeeds entirely, or it fails entirely.

Atomicity is particularly important when we want to atomically write more than one record in the same transaction. The raise/2 function, shown as an example above, writes one record only. The insert_emp/3 function, shown in the program listing in Chapter 2, writes the record employee as well as employee relations such as at_dep and in_proj into the database. If we run this latter code inside a transaction, then the transaction handler ensures that the transaction either succeeds completely, or not at all.

Mnesia is a distributed DBMS where data can be replicated on several nodes. In many such applications, it is important that a series of write operations are performed atomically inside a transaction. The atomicity property ensures that a transaction take effect on all nodes, or none at all.

4.1.2 Consistency

Consistency. This transaction property ensures that a transaction always leaves the DBMS in a consistent state. For example, Mnesia ensures that inconsistencies will not occur if Erlang, Mnesia or the computer crashes while a write operation is in progress.

4.1.3 Isolation

Isolation. This transaction property ensures that transactions which execute on different nodes in a network, and access and manipulate the same data records, will not interfere with each other.

The isolation property makes it possible to concurrently execute the raise/2 function. A classical problem in concurrency control theory is the so called "lost update problem".

The isolation property is extremely useful if the following circumstances occurs where an employee (with an employee number 123) and two processes, (P1 and P2), are concurrently trying to raise the salary for the employee. The initial value of the employees salary is, for example, 5. Process P1 then starts to execute, it reads the employee record and adds 2 to the salary. At this point in time, process P1 is for some reason preempted and process P2 has the opportunity to run. P2 reads the record, adds 3 to the salary, and finally writes a new employee record with the salary set to 8. Now, process P1 start to run again and writes its employee record with salary set to 7, thus effectively overwriting and undoing the work performed by process P2. The update performed by P2 is lost.

A transaction system makes it possible to concurrently execute two or more processes which manipulate the same record. The programmer does not need to check that the updates are synchronous, this is overseen by the transaction handler. All programs accessing the database through the transaction system may be written as if they had sole access to the data.

4.1.4 Durability

Durability. This transaction property ensures that changes made to the DBMS by a transaction are permanent. Once a transaction has been committed, all changes made to the database are durable - i.e. they are written safely to disc and will not be corrupted or disappear.

Note!

The durability feature described does not entirely apply to situations where Mnesia is configured as a "pure" primary memory database.

4.2 Locking

Different transaction managers employ different strategies to satisfy the isolation property. Mnesia uses the standard technique of two-phase locking. This means that locks are set on records before they are read or written. Mnesia uses five different kinds of locks.

Mnesia employs a strategy whereby functions such as mnesia:read/1 acquire the necessary locks dynamically as the transactions execute. Mnesia automatically sets and releases the locks and the programmer does not have to code these operations.

Deadlocks can occur when concurrent processes set and release locks on the same records. Mnesia employs a "wait-die" strategy to resolve these situations. If Mnesia suspects that a deadlock can occur when a transaction tries to set a lock, the transaction is forced to release all its locks and sleep for a while. The Fun in the transaction will be evaluated one more time.

For this reason, it is important that the code inside the Fun given to mnesia:transaction/1 is pure. Some strange results can occur if, for example, messages are sent by the transaction Fun. The following example illustrates this situation:

bad_raise(Eno, Raise) ->
    F = fun() ->
                [E] = mnesia:read({employee, Eno}),
                Salary = E#employee.salary + Raise,
                New = E#employee{salary = Salary},
                io:format("Trying to write ... ~n", []),
                mnesia:write(New)
        end,
    mnesia:transaction(F).

This transaction could write the text "Trying to write ... " a thousand times to the terminal. Mnesia does guarantee, however, that each and every transaction will eventually run. As a result, Mnesia is not only deadlock free, but also livelock free.

The Mnesia programmer cannot prioritize one particular transaction to execute before other transactions which are waiting to execute. As a result, the Mnesia DBMS transaction system is not suitable for hard real time applications. However, Mnesia contains other features that have real time properties.

Mnesia dynamically sets and releases locks as transactions execute, therefore, it is very dangerous to execute code with transaction side-effects. In particular, a receive statement inside a transaction can lead to a situation where the transaction hangs and never returns, which in turn can cause locks not to release. This situation could bring the whole system to a standstill since other transactions which execute in other processes, or on other nodes, are forced to wait for the defective transaction.

If a transaction terminates abnormally, Mnesia will automatically release the locks held by the transaction.

We have shown examples of a number of functions that can be used inside a transaction. The following list shows the simplest Mnesia functions that work with transactions. It is important to realize that these functions must be embedded in a transaction. If no enclosing transaction (or other enclosing Mnesia activity) exists, they will all fail.

4.2.1 Sticky Locks

As previously stated, the locking strategy used by Mnesia is to lock one record when we read a record, and lock all replicas of a record when we write a record. However, there are applications which use Mnesia mainly for its fault-tolerant qualities, and these applications may be configured with one node doing all the heavy work, and a standby node which is ready to take over in case the main node fails. Such applications may benefit from using sticky locks instead of the normal locking scheme.

A sticky lock is a lock which stays in place at a node after the transaction which first acquired the lock has terminated. To illustrate this, assume that we execute the following transaction:

        F = fun() ->
              mnesia:write(#foo{a = kalle})
            end,
        mnesia:transaction(F).
      

The foo table is replicated on the two nodes N1 and N2.
Normal locking requires:

If we use sticky locks, we must first change the code as follows:

        F = fun() ->
              mnesia:s_write(#foo{a = kalle})
            end,
        mnesia:transaction(F).
      

This code uses the s_write/1 function instead of the write/1 function. The s_write/1 function sets a sticky lock instead of a normal lock. If the table is not replicated, sticky locks have no special effect. If the table is replicated, and we set a sticky lock on node N1, this lock will then stick to node N1. The next time we try to set a sticky lock on the same record at node N1, Mnesia will see that the lock is already set and will not do a network operation in order to acquire the lock.

It is much more efficient to set a local lock than it is to set a networked lock, and for this reason sticky locks can benefit application that use a replicated table and perform most of the work on only one of the nodes.

If a record is stuck at node N1 and we try to set a sticky lock for the record on node N2, the record must be unstuck. This operation is expensive and will reduce performance. The unsticking is done automatically if we issue s_write/1 requests at N2.

4.2.2 Table Locks

Mnesia supports read and write locks on whole tables as a complement to the normal locks on single records. As previously stated, Mnesia sets and releases locks automatically, and the programmer does not have to code these operations. However, transactions which read and write a large number of records in a specific table will execute more efficiently if we start the transaction by setting a table lock on this table. This will block other concurrent transactions from the table. The following two function are used to set explicit table locks for read and write operations:

Alternate syntax for acquisition of table locks is as follows:

        mnesia:lock({table, Tab}, read)
        mnesia:lock({table, Tab}, write)
      

The matching operations in Mnesia may either lock the entire table or just a single record (when the key is bound in the pattern).

4.2.3 Global Locks

Write locks are normally acquired on all nodes where a replica of the table resides (and is active). Read locks are acquired on one node (the local one if a local replica exists).

The function mnesia:lock/2 is intended to support table locks (as mentioned previously) but also for situations when locks need to be acquired regardless of how tables have been replicated:

        mnesia:lock({global, GlobalKey, Nodes}, LockKind)

        LockKind ::= read | write | ...
        

The lock is acquired on the LockItem on all Nodes in the nodes list.

4.3 Dirty Operations

In many applications, the overhead of processing a transaction may result in a loss of performance. Dirty operation are short cuts which bypass much of the processing and increase the speed of the transaction.

Dirty operation are useful in many situations, for example in a datagram routing application where Mnesia stores the routing table, and it is time consuming to start a whole transaction every time a packet is received. For this reason, Mnesia has functions which manipulate tables without using transactions. This alternative to processing is known as a dirty operation. However, it is important to realize the trade-off in avoiding the overhead of transaction processing:

The major advantage of dirty operations is that they execute much faster than equivalent operations that are processed as functional objects within a transaction.

Dirty operations are written to disc if they are performed on a table of type disc_copies, or type disc_only_copies. Mnesia also ensures that all replicas of a table are updated if a dirty write operation is performed on a table.

A dirty operation will ensure a certain level of consistency. For example, it is not possible for dirty operations to return garbled records. Hence, each individual read or write operation is performed in an atomic manner.

All dirty functions execute a call to exit({aborted, Reason}) on failure. The following functions are available:

4.4 Record names versus table names

In Mnesia, all records in a table must have the same name. All the records must be instances of the same record type. The record name does however not necessarily be the same as the table name. Even though that it is the case in the most of the examples in this document. If a table is created without the record_name property the code below will ensure all records in the tables have the same name as the table:

      mnesia:create_table(subscriber, [])
    

However, if the table is is created with an explicit record name as argument, as shown below, it is possible to store subscriber records in both of the tables regardless of the table names:

      TabDef = [{record_name, subscriber}],
      mnesia:create_table(my_subscriber, TabDef),
      mnesia:create_table(your_subscriber, TabDef).
    

In order to access such tables it is not possible to use the simplified access functions as described earlier in the document. For example, writing a subscriber record into a table requires a mnesia:write/3function instead of the simplified functions mnesia:write/1 and mnesia:s_write/1:

      mnesia:write(subscriber, #subscriber{}, write)
      mnesia:write(my_subscriber, #subscriber{}, sticky_write)
      mnesia:write(your_subscriber, #subscriber{}, write)
    

The following simplified piece of code illustrates the relationship between the simplified access functions used in most examples and their more flexible counterparts:

      mnesia:dirty_write(Record) ->
        Tab = element(1, Record),
        mnesia:dirty_write(Tab, Record).
      
      mnesia:dirty_delete({Tab, Key}) ->
        mnesia:dirty_delete(Tab, Key).
      
      mnesia:dirty_delete_object(Record) ->
        Tab = element(1, Record),
        mnesia:dirty_delete_object(Tab, Record) 
      
      mnesia:dirty_update_counter({Tab, Key}, Incr) ->
        mnesia:dirty_update_counter(Tab, Key, Incr).
      
      mnesia:dirty_read({Tab, Key}) ->
        Tab = element(1, Record),
        mnesia:dirty_read(Tab, Key).
      
      mnesia:dirty_match_object(Pattern) ->
        Tab = element(1, Pattern),
        mnesia:dirty_match_object(Tab, Pattern).
      
      mnesia:dirty_index_match_object(Pattern, Attr) 
        Tab = element(1, Pattern),
        mnesia:dirty_index_match_object(Tab, Pattern, Attr).
      
      mnesia:write(Record) ->
        Tab = element(1, Record),
        mnesia:write(Tab, Record, write).
      
      mnesia:s_write(Record) ->
        Tab = element(1, Record),
        mnesia:write(Tab, Record, sticky_write).
      
      mnesia:delete({Tab, Key}) ->
        mnesia:delete(Tab, Key, write).
      
      mnesia:s_delete({Tab, Key}) ->
        mnesia:delete(Tab, Key, sticky_write).
      
      mnesia:delete_object(Record) ->
        Tab = element(1, Record),
        mnesia:delete_object(Tab, Record, write).
      
      mnesia:s_delete_object(Record) ->
        Tab = element(1, Record),
        mnesia:delete_object(Tab, Record. sticky_write).
      
      mnesia:read({Tab, Key}) ->
        mnesia:read(Tab, Key, read).
      
      mnesia:wread({Tab, Key}) ->
        mnesia:read(Tab, Key, write).
      
      mnesia:match_object(Pattern) ->
        Tab = element(1, Pattern),
        mnesia:match_object(Tab, Pattern, read).
      
      mnesia:index_match_object(Pattern, Attr) ->
        Tab = element(1, Pattern),
        mnesia:index_match_object(Tab, Pattern, Attr, read).
    

4.5 Activity concept and various access contexts

As previously described, a functional object (Fun) performing table access operations as listed below may be passed on as arguments to the function mnesia:transaction/1,2,3:

These functions will be performed in a transaction context involving mechanisms like locking, logging, replication, checkpoints, subscriptions, commit protocols etc.However, the same function may also be evaluated in other activity contexts.
The following activity access contexts are currently supported:

By passing the same "fun" as argument to the function mnesia:async_dirty(Fun [, Args]) it will be performed in dirty context. The function calls will be mapped to the corresponding dirty functions. This will still involve logging, replication and subscriptions but there will be no locking, local transaction storage or commit protocols involved. Checkpoint retainers will be updated but will be updated "dirty". Thus, they will be updated asynchronously. The functions will wait for the operation to be performed on one node but not the others. If the table resides locally no waiting will occur.

By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun [, Args]) it will be performed in almost the same context as mnesia:async_dirty/1,2. The difference is that the operations are performed synchronously. The caller will wait for the updates to be performed on all active replicas. Using sync_dirty is useful for applications that are executing on several nodes and want to be sure that the update is performed on the remote nodes before a remote process is spawned or a message is sent to a remote process. This is also useful in situations where an application performs frequent or voluminous updates which may overload Mnesia on other nodes.

Mnesia tables with storage type RAM_copies and disc_copies are implemented internally as "ets-tables" and it is possible for applications to access the these tables directly. This is only recommended if all options have been weighed and the possible outcomes are understood. By passing the earlier mentioned "fun" to the function mnesia:ets(Fun [, Args]) it will be performed but in a very raw context. The operations will be performed directly on the local ets tables assuming that the local storage type are RAM_copies and that the table is not replicated on other nodes. Subscriptions will not be triggered nor checkpoints updated, but this operation is blindingly fast. Disc resident tables should not be updated with the ets-function since the disc will not be updated.

The Fun may also be passed as an argument to the function mnesia:activity/2,3,4 which enables usage of customized activity access callback modules. It can either be obtained directly by stating the module name as argument or implicitly by usage of the access_module configuration parameter. A customized callback module may be used for several purposes, such as providing triggers, integrity constraints, run time statistics, or virtual tables.
The callback module does not have to access real Mnesia tables, it is free to do whatever it likes as long as the callback interface is fulfilled.
In Appendix C "The Activity Access Call Back Interface" the source code for one alternate implementation is provided (mnesia_frag.erl). The context sensitive function mnesia:table_info/2 may be used to provide virtual information about a table. One usage of this is to perform Mnemosyne queries within an activity context with a customized callback module. By providing table information about table indices and other Mnemosyne requirements, Mnemosyne may be used as a generic query language to access virtual tables.

Mnemosyne queries may be performed in all these activity contexts (transaction, async_dirty, sync_dirty and ets). The ets activity will only work if the table has no indices.

Note!

The mnesia:dirty_* function always executes with async_dirty semantics regardless of which activity access contexts are invoked. They may even invoke contexts without any enclosing activity access context.

4.6 Nested transactions

Transactions may be nested in an arbitrary fashion. A child transaction must run in the same process as its parent. When a child transaction aborts, the caller of the child transaction will get the return value {aborted, Reason} and any work performed by the child will be erased. If a child transaction commits, the records written by the child will be propagated to the parent.

No locks are released when child transactions terminate. Locks created by a sequence of nested transactions are kept until the topmost transaction terminates. Furthermore, any updates performed by a nested transaction are only propagated in such a manner so that the parent of the nested transaction sees the updates. No final commitment will be done until the top level transaction is terminated. So, although a nested transaction returns {atomic, Val}, if the enclosing parent transaction is aborted, the entire nested operation is aborted.

The ability to have nested transaction with identical semantics as top level transaction makes it easier to write library functions that manipulate mnesia tables.

Say for example that we have a function that adds a new subscriber to a telephony system:

      add_subscriber(S) ->
          mnesia:transaction(fun() ->
              case mnesia:read( ..........
    

This function needs to be called as a transaction. Now assume that we wish to write a function that both calls the add_subscriber/1 function and is in itself protected by the context of a transaction. By simply calling the add_subscriber/1 from within another transaction, a nested transaction is created.

4.7 Pattern matching

Mnesia provides the programmer with several functions for matching records against a pattern. The most useful function of these is:

      mnesia:match_object(Tab, Pattern, LockKind) ->
         transaction abort | RecordList
    

This function matches a Pattern against all records in table Tab. It is not necessarily performed as an exhaustive search of the entire table. By utilizing indices and bound values in the pattern, the actual work done by the function can be condensed into a few hash lookups.

The pattern provided to the mnesia:match_object/3 function must be a valid record, and the first element of the provided tuple must be the record_name of the table. The special element '_' matches any data structure in Erlang (otherwise known as an Erlang term).

Use the function mnesia:table_info(Tab, wild_pattern) to obtain a basic pattern which matches all records in a table. Do not make the pattern hardcore since it will make your code more vulnerable to future changes of the record definition.

For the employee table the wild pattern will look like:

      {employee, '_', '_', '_', '_', '_',' _'}.
    

In order to constrain the match you must replace some of the '_' elements. The code for matching out all female employees, looks like:

      WildPat = mnesia:table_info(employee, wild_pattern),
      Pat = WildPat#employee{sex = female},
      F = fun() -> mnesia:match_object(Pat) end,
      Females = mnesia:transaction(F).
    

It is also possible to use the match function if we want to check the equality of different attributes. Assume that we want to find all employees which happens to have a employee number which equals to their room number:

      WildPat = mnesia:table_info(employee, wild_pattern),
      Pat = WildPat#employee{emp_no = '$1', room_no = '$1'},
      F = fun() -> mnesia:match_object(Pat) end,
      Odd = mnesia:transaction(F).
    

If the key attribute is bound in a pattern, the match operation is very efficient. However, if the key attribute in a pattern is given as '_', or '$1', the whole employee table must be searched for records that match. If the table is large, this can become a time consuming operation. This can be remedied with indices (refer to Chapter 5: Indexing).

The function mnesia:all_keys(Tab) returns all keys in a table.


Copyright © 1991-2000 Ericsson Utvecklings AB