Getting started with Erlang

The original article and examples was written by Claes Wikstrom in 1998. In april 2008, We removed an example that no longer works and revised the examples to eliminate warnings and use the modern type tests (when is_list(List) instead of when list(List)).

In this document I aim to provide a number of small examples of nice Erlang programs. The goal of these examples is to let the reader get started immediataly doing some interesting stuff with the erlang programming environment. The document assumes some knowledge about erlang programming in general.

Programs in this document

count_chars.erl

Count the number of 'x' chars in a file. Different versions.

wc.erl

A small wc look alike.

find.erl

A small find look alike.

slogger.erl

A simple term logger. There is also a small example.

glogger.erl

The same logger again with the gen_server module.

klib.erl

My personal goodies library.

time_server.erl

A TCP/IP time server.

chargen.erl

A threaded chargen server.

ftpd.erl

A complete ftpd.

gsmines.erl

The boring game of mines. Done with the library gs.

We start off with some examples of file handling. Most off the stuff related to file handling resides in the module called file. The easist method of finding out which functions that are available in the file module is to read the manual page. The manual page is available on UNIX systems by invoking the command % erl -man file at the UNIX shell prompt. This of course assumes a correctly installed Erlang system. On a Windows box, there is no manual reader but exactly the same information is available in HTML format.

Anyway, let's get started with some file IO.

File IO

Counting x'es

%%%----------------------------------------------------------------------
%%% File    : count_chars.erl
%%% Author  : Claes Wikstrom <klacke@bluetail.com>
%%% Purpose : Count the x chars in a file
%%% Created : 20 Oct 1998 by Claes Wikstrom <klacke@bluetail.com>
%%%----------------------------------------------------------------------

-module(count_chars).
-author('klacke@bluetail.com').

-export([file/1]).

file(Fname) ->
    case file:open(Fname, [read, raw, binary]) of
	{ok, Fd} ->
	    scan_file(Fd, 0, file:read(Fd, 1024));
	{error, Reason} ->
	    {error, Reason}
    end.

scan_file(Fd, Occurs, {ok, Binary}) ->
    scan_file(Fd, Occurs + count_x(Binary), file:read(Fd, 1024));
scan_file(Fd, Occurs, eof) ->
    file:close(Fd),
    Occurs;
scan_file(Fd, _Occurs, {error, Reason}) ->
    file:close(Fd),
    {error, Reason}.

The file/1 function opens the file and reads the characters in chunks of 1 k. For each chunk it calls a function count_x/1 to do the real counting. This function transforms each binary to a list of characters in order to be able to traverse and count. We have:

count_x(Bin) ->
    count_x(binary_to_list(Bin), 0).
count_x([$x|Tail], Ack) ->
    count_x(Tail, Ack+1);
count_x([_|Tail], Ack) ->
    count_x(Tail, Ack);
count_x([], Ack) ->
    Ack.

Now to compile and run this code we invoke the erlang system at the unix prompt and enter the following commands:

% erl
Erlang (JAM) emulator version 4.7.3
 
Eshell V4.7.3  (abort with ^G)
1> c:c(count_chars).
{ok,count_chars}
2> count_chars:file("count_chars.erl").
17
3> 

So the file "count_chars.erl" contains 17 x'es.

The abovementioned flaw is that we do not close the file in the same function where we opened the file. In this particular case we close it in the function just below, but it is a general good rule to release resources in the source code where they are allocated. One way to rectify this here is to rewrite the code like:

file(Fname) ->
    case file:open(Fname, [read, raw, binary]) of
	{ok, Fd} ->
	    Res = scan_file(Fd, 0, file:read(Fd, 1024)),
	    file:close(Fd),
            Res;
         ....

However we can do better, we can write a general purpose function with_file which feed a user provided Fun with chunks of data until done.

General purpose file IO

The technique of generalizing a common pattern of functionality into a framework which uses higher order functions to do the work is a powerful lines-of-code saver. So the general purpose function is:

with_file(File, Fun, Initial) ->
    case file:open(File, [read, raw, binary]) of
	{ok, Fd} ->
	    Res = feed(Fd, file:read(Fd, 1024), Fun, Initial),
	    file:close(Fd),
	    Res;
	{error, Reason} ->
	    {error, Reason}
    end.

feed(Fd, {ok, Bin}, Fun, Farg) ->
    case Fun(Bin, Farg) of
	{done, Res} ->
	    Res;
	{more, Ack} ->
	    feed(Fd, file:read(Fd, 1024), Fun, Ack)
    end;
feed(Fd, eof, Fun, Ack) ->
    Ack;
feed(_Fd, {error, Reason}, _Fun, _Ack) ->
    {error, Reason}.

The user provides three arguments to the function.

File

the name of the file to work on.

Fun

a functional object which must return either of {more, Ack} or {done, Res} in order to guide the feed loop what to do.

Initial

the initial ackumulator parameter to the fun.

Now the original count_chars:file/1 function becomes much shorter.

file1(File) ->
    F = fun(Bin, Int) -> 
		{more, count_x(Bin) + Int}
	end,
    klib:with_file(File, F, 0).

As a matter of fact we can do even better than that, we can use a function defined in "lists.erl" which folds over a list. This is admittedly not code that ought to be in the first section of a beginner guide but we provide it anyway. We have:

file2(File) ->
    {ok,B} = file:read_file(File),
    lists:foldl(fun($x, Ack) ->
			1 + Ack;
		   (_, Ack) ->
			Ack
		end, 0, binary_to_list(B)).

Now we have three version of the same function say we want to meassure how fast they are. The mudule "timer" has a function called "tc(Mod, Fun, Args)" which executes a function and times it. Lets do it at the shell prompt:

32> timer:tc(count_chars, file, ["index.html"]).
{59554,68}
33>  timer:tc(count_chars, file1, ["index.html"]).
{69939,68}
34> timer:tc(count_chars, file2, ["index.html"]).
{335579,68}

The tc/3 function returns a tuple {MicroSeconds, Result where MicroSeconds is the number of micro seconds it took to eavluate the function and Result is the avaluation result.

We see that the first (and most complicated) version is the fastest. It takes 63 milli seconds whereas the next version which uses the with_file/3 function takes 69 milli seconds. On the other hand the most beautiful version, file2/1 which folds over the entire lists takes an awful 337 milli seconds.

Word count

wc:file(File)
wc:files(FileList)

-module(wc).

-import(lists, [map/2, foreach/2]).
-export([file/1, files/1]).

file(File) ->
    output([gfile(File)]).

gfile(File) ->
    Fun = fun(Bin, Count) ->
		  count_bin(binary_to_list(Bin), inspace, Count)
	  end,
    {File, klib:with_file(File, Fun, {0,0,0})}.


count_bin([H|T], Where, {C,W,L}) ->
    case classify_char(H) of
	newline  when Where == inspace ->
	    count_bin(T, inspace, {C+1, W, L+1});
	newline when Where == inword ->
	    count_bin(T, inspace, {C+1, W+1, L+1});
	space  when Where == inspace ->
	    count_bin(T, inspace, {C+1, W, L});
	space  when Where == inword ->
	    count_bin(T, inspace, {C+1, W+1, L});
	char ->
	    count_bin(T, inword, {C+1, W, L})
    end;
count_bin([], inword, {C, W, L}) ->
    {more, {C, W+1, L}};
count_bin([], inspace, {C, W, L}) ->
    {more, {C, W, L}}.


classify_char($ ) ->
    space;
classify_char($\t) ->
    space;
classify_char($\n) ->
    newline;
classify_char(_) ->
    char.

files(Files) ->
    output(map(fun(F) -> gfile(F) end, Files)).

output(Counts) ->
    io:format("~-25s ~-10s ~-10s ~-10s~n",
	      ["file", "chars", "words", "lines"]),
    foreach(fun({File, {C,W,L}}) ->
		    ok = io:format("~-25s ~-10w ~-10w ~-10w~n", 
			      [File, C, W, L])
	    end, Counts).

60> {ok, L} = file:list_dir("."), wc:files(L).
file                      chars      words      lines     
wc.beam                   2079       40         21        
count_chars.beam          1796       31         22        
klacke_ex.html~           6173       897        219       
test                      11         3          1         
wc.erl~                   587        46         27        
klacke_ex.html.orig       6173       897        219       
klacke_ex.html            7642       1079       286       
wc.erl                    1661       176        73        
count_chars.erl           1767       198        87        

Finding files

find:files("/home/klacke", 
           ".*\.erl", fun(F) -> {File, c:c(File)} end)

• A top directory where to start the search
• A regular expression that must match the files.
• A Fun to perform some action on the files we find.

-module(find).
-author('klacke@bluetail.com').
-include_lib("kernel/include/file.hrl").

-export([files/3]).

files(Top, Re, Action) ->
    case file:list_dir(Top) of
	{ok, Files} ->
	    files(Top, Files, Re, Action, []);
	{error, Reason}  ->
	    {error, {Top, Reason}}
    end.

files(Top, [F|Tail], Re, Action, Ack) ->
    F2 = Top ++ "/" ++ F,
    case file:read_file_info(F2) of
	{ok, FileInfo} when FileInfo#file_info.type == directory ->
	    case files(F2, Re, Action) of
		{error, Reason} ->
		    {error, Reason};
		List ->
		    files(Top, Tail, Re, Action, List ++ Ack)
	    end;
	{error, Reason} ->
	    {error, {F2, Reason}};
	{ok, FileInfo} when FileInfo#file_info.type == regular ->
	    case catch regexp:match(F, Re) of
		{match, _,_} ->
		    files(Top, Tail, Re, Action, [Action(F2) | Ack]);
		nomatch ->
		    files(Top, Tail, Re, Action, Ack);
		{error, Reason} ->
		    {error, {F2, {regexp, Reason}}}
	    end;
	Other ->
	    files(Top, Tail, Re, Action, Ack)
    end;

files(_Top, [], _Re, _Action, Ack) ->
    Ack.
    

A simple term logger

In this final section on file IO we provide a simple term logger that writes Erlang terms to a file.

The BIF term_to_binary/1 produces a binary data object from any term. Such a binary can be written to a file. The reverse operation is binary_to_term/1 which can be used to reproduce the orignal term. The format we choose to have on the file is to prepend each term with a four (4) byte length field in order to indicate the length of the actual term. The logger runs as a separate process and we shall provide two different versions of the logger, one written with plain Erlang and the other by utilizing the generic server gen_server module. One function which is used by the logger is the function of transforming an integer to a four byte list and vice versa. This function is added to the klib.erl library. Here is the code:

i32(B) when is_binary(B) ->
    i32(binary_to_list(B, 1, 4));
i32([X1, X2, X3, X4]) ->
    (X1 bsl 24) bor (X2 bsl 16) bor (X3 bsl 8) bor X4;
i32(Int) when integer(Int) ->
    [(Int bsr 24) band 255,
     (Int bsr 16) band 255,
     (Int bsr  8) band 255,
     Int band 255].

It is also nice to be able to read an integer from a file, the code in klib.erl is:

getint32(F) ->
    {ok, B} = file:read(F, 4),
    i32(B).

-module(slogger).
-author('klacke@bluetail.com').

-export([start/0, start/1, stop/0, log/1, upread/1, truncate/0]).

We need to be able to stop and start the server. The client functions are:

log/1

Log a term to the end of the file.

truncate/0

Truncate the log file.

upread/1

Read all the terms in the logfile and apply Fun to each and every term.

-define(LOGFILE, "slog.log").

start() ->
    start(?LOGFILE).

start(F) ->
    case whereis(?MODULE) of
	undefined ->
	    register(?MODULE, spawn(?MODULE, loop0, [F]));
	Pid ->
	    true
    end.

stop()->
    req(stop).

req(R) ->
    ?MODULE ! {self(), R},
    receive
	{?MODULE, Reply} ->
	    Reply
    end.

The loop function that is spawned by the the start function has two part, an initialization part and a loop part. The coe is:

loop0(FileName) ->
    case file:open(FileName, [read, write, raw, binary]) of
	{ok, Fd} ->
	    {ok, Eof} = file:position(Fd, eof),
	    file:position(Fd, bof),
	    FilePos = position_fd(Fd, 0),
	    maybe_warn(FilePos, Eof),
	    loop(Fd);
	{error, Reason} ->
	    exit(Reason)
    end.

maybe_warn(FilePos, Eof) ->
    if
	FilePos == Eof ->
	    ok;
	true ->
	    warn("~w bytes truncated \n", 
		 [Eof - FilePos])
    end.
	    

We have added some extra code to check the logfile when it is opened. We need to position the file descriptor to the end of the logfile. To do that we could have called file:position(Fd, eof), however we need to cater for the case where the last logger term in the previous section was corrupted due to an interrupted write operation. Thw function that does the check is:

position_fd(Fd, LastPos) ->
    case catch getint32(Fd) of
	Int when is_integer(Int) ->
	    case file:read(Fd, Int) of
		{ok, B} when size(B) ==  Int ->
		    position_fd(Fd, LastPos + 4 + Int);
		_ ->
		    file:position(Fd, LastPos),
		    file:truncate(Fd)
	    end;
	_ ->
	    file:position(Fd, LastPos),
	    file:truncate(Fd),
	    LastPos
    end.

The position_fd/2 function returns last file position that was ok. If this is equal to the physical file position, all is ok. So, we are approaching the real code that does the actual logging work. The loop/1 server loop.

loop(Fd) ->		    
    receive
	{From, {log, Bin}} ->
	    From ! {?MODULE, log_binary(Fd, Bin)};
	{From, {upread, Fun}} ->
	    From ! {?MODULE, upread(Fd, Fun)};
	{From, truncate} ->
	    file:position(Fd, bof),
	    file:truncate(Fd),
	    From ! {?MODULE, ok};
	{From, stop} ->	
	    file:close(Fd),
	    From ! {?MODULE, stopped},
	    exit(normal)
    end,
    loop(Fd).

The truncate and the stop requests are handled immediataly in the loop whereas the request to log and upread are handled by special help functions. First we have the log_binary/2 function:

log_binary(Fd, Bin) ->
    Sz = size(Bin),
    case file:write(Fd, [i32(Sz), Bin]) of
	ok ->
	    ok;
	{error, Reason} ->
	    warn("Cant't write logfile ~p ", [Reason]),
	    {error, Reason}
    end.
    

The only noticeable thing here is the type of the structure that is passed to file:write/2 for IO. It is a list on the form [[int, int, int, int], binary]. This structure will be flattened by the port. This applies to all ports and if the Port supports the so called writev() interface (which is documented elsewhere) the writev() function of the native operating system will be called with an array length of 2 where the first array will hold the four bytes produced by the call to i32() and the second array will hold the actual binary data.

Finally the function to do the upreading, i.e a function that blocks the server for logging and then traverses the log and applies a function to each and every logged item is:

upread(Fd, Fun) ->		
    {ok, Curr} = file:position(Fd, cur),
    file:position(Fd, bof),
    upread(Fd, catch get_term(Fd), Fun).

upread(Fd, {'EXIT', _}, Fun) ->
    ok;
upread(Fd, Term, Fun) ->
    Fun(Term),
    upread(Fd, catch get_term(Fd), Fun).
    
    
get_term(Fd) ->
    I = getint32(Fd),
    {ok, B} = file:read(Fd, I),
    binary_to_term(B).

And finally the client functions to access the server:

upread(Fun) ->
    req({upread, Fun}).

truncate() ->
    req(truncate).

log(Term) ->
    req({log, term_to_binary(Term)}).

A typical little caveat here with the upread/1 function is that the client function is sitting in a receive statement waiting for upread/1 to perform its work. It is thus not possible to have the upread/1 function send all the terms to the client. (Since the client is allready suspended). To do that, we need an auxilliary process. Here is a little example session at the shell prompt:

2> slogger:start().
true
3> slogger:log({abc, "cba"}).
ok
4> slogger:log(code:which(slogger)).  
ok
5> slogger:upread(fun(X) ->  io:format("~p~n", [X])  end).
{abc,"cba"}
"/home/super/klacke/doc/examples/slogger.jam"
ok
6> 

A simple term logger (again)

gen_servers are extremely powerful and easy to use. The absolutely easiest way to write a gen server is to invoke the gen_server skeleton from the emacs mode while editing.

The entire framework of a "do nothing" gen server is generated and we just fill in the details. This way we get all the goodies that come from gen servers in general. They can be upgraded and downgraded wile running, they can be debugged and traced and they fit into the general application concept of Erlang.

The code is available in glogger.erl.

Socket IO

We start with a simple time server, it receives requests on TCP port 2345 and replies with a string indicating the current time.

A time server

The complete code is here and it is extremely simple. It just opens a socket a replies with the current data and time to anyone that connects to the server.

We have:

start(Pno) ->
    spawn(?MODULE, loop0, [Pno]).

loop0(Port) ->
    case gen_tcp:listen(Port, [binary, {packet, 0}, {active, false}]) of
	{ok, LSock} ->
	    loop(LSock);
	_ ->
	    stop
    end.

loop(Listen) ->
    case gen_tcp:accept(Listen) of
	{ok, S} ->
	    gen_tcp:send(S, io_lib:format("~p~n", [{date(), time()}])),
	    gen_tcp:close(S),
	    loop(Listen);
	_ ->
	    loop(Listen)
    end.

This is is a so called iterative server. It finishes each request until it accepts a new request. Another type of TCP/IP server creates a new process for each request. This is known as a concurrent or a threaded server. As we can expect erlang is well suited for the implementation of threaded servers.

A threaded chargen server

One of the most strange servers on many UNIX machines is the chargen server. It typically resides on port 19 and replies with an endless stream of printable ASCII characters when connected to.

It can thus act as an indefinite source of network input, typically used to test various applications.

Nevertheless, this type of server need to be threaded. Thus we provide an example here with a threaded chargen server.

The best way to write a threaded TCP server in Erlang is to first create the listen socket, then create a new worker process which does the call to accept. Once the accept is complete the worker process notifies the servere about this and the server can create yet another worker process. First we need then initial startup code:

-define(PORTNO, 2019).

start_link() ->
    start_link(?PORTNO).
start_link(P) ->
    spawn_link(?MODULE, loop0, [P]).

loop0(Port) ->
    case gen_tcp:listen(Port, [binary, {reuseaddr, true},
			       {packet, 0}, {active, false}]) of
	{ok, LSock} ->
	    spawn(?MODULE, worker, [self(), LSock]),
	    loop(LSock);
	Other ->
	    io:format("Can't listen to socket ~p~n", [Other])
    end.

This code spawns off the initial worker process and goes into a loop. The loop is simple:

loop(S) ->
    receive
	next_worker ->
	    spawn_link(?MODULE, worker, [self(), S])
    end,
    loop(S).

It listens for next_worker messages and spawns off a new worker for each such message. This way we are guaranteed to always have exactly one worker waiting to accept on the socket. The worker process accepts the socket and start to generate characters. We have:

worker(Server, LS) ->
    case gen_tcp:accept(LS) of
	{ok, Socket} ->
	    Server ! next_worker,
	    gen_chars(Socket, 32);
	{error, Reason} ->
	    Server ! next_worker,
	    io:format("Can't accept socket ~p~n", [Reason])
    end.


gen_chars(Socket, Char) ->
    Line = make_line(Char, 0),
    case gen_tcp:send(Socket, Line) of
	{error, Reason} -> exit(normal);
	ok -> gen_chars(Socket, upchar(Char))
    end.


make_line(Char, 70) ->
    [10];
make_line(127, Num) ->
    make_line(32, Num);
make_line(Char, Num) ->
    [Char | make_line(Char+1, Num+1)].

upchar(Char) ->
    if
	Char + 70 > 127 ->
	    32 + (127 - Char);
	true ->
	    Char + 70
    end.

Name server

We continue with a simple name server, a server that sits and listens for TCP request on a port and then either responds to "GET" requests or "PUT" request. We want the server to be threaded so we have a similar setup phase as the chargen code.

Socket server framework

The chargen server from the previous section exhibits a behaviour which is truly typical for many internet servers. Many many tcp/ip servers have the typical sequence of listen followed by a loop of accept calls. We can parametrize this behaviour in a manner simmilar to the with_file/2 function from the first section. We want to have a function called with_socket/2 which takes a Portnumber and a Fun as parameters and the runs a standard tcp/ip server on the port number and applies Fun in a separate process to all requests. This is indeed possible and the code for the with_socket/2 function (ofcourse) resides in klib.erl. This code has some more luggage than the previous chargen server, for example we have client functions to query the server about it's status as so on. The code explained:

with_socket(Port, Fun) ->
    spawn(?MODULE, with_socket0, [Port, Fun]).

with_socket0(Port, Fun) ->
    process_flag(trap_exit, true),
    case gen_tcp:listen(Port, [binary, {packet, 0}, 
			       {reuseaddr, true},
			       {active, false}]) of
	{ok, LSock} ->
	    P = spawn_link(?MODULE, sock_handler, [self(), Fun, LSock]),
	    sock_loop(P, Fun, 1, nil, LSock);
	Other ->
	    Other
    end.

An ftp daemon

In this section we introduce a complete implementation of an ftp daemon according to RFC 765. We don't comment on the code merely provide it. The server is almost 1000 lines of Erlang code. The code is at ftpd.erl

Gui programming

We provide the boring game of mines as an example of an gs based program: gsmines.erl