Lux IO is a yet another fast database manager. It supports B+-tree and Array index in either cluster or non-cluster index.
It's originally designed for storing large expanding data as a value in Lux Search Engine, but it's also pretty fast for small and a large number of data.

• Fast key lookup (B+-tree, Array)
• Support both clustered and non-clustered index
• mmap(2) the whole index structure in clustered index
• 2 ways in data management in non-clustered index (Linked and Padded)
• Linked mode for expanding data
• Padded mode for fixed-sized data

Benchmark tests with another fast database managers Berkeley DB(4.7.25) and Tokyo Cabinet(1.3.12).
(OS version and machine spec for benchmarks are CentOS(2.6.18-92.1.13.el5xen) and Intel(R) Core(TM)2 Duo CPU E7200 @ 2.53GHz 4GB Memory)

Test cases are listed as follows. (you can check the programs in ptest/single directory.)

• sequential insert/select on 1 million records
• key range is "00000001" to "01000000"(8bytes) and value range is 1 to 1000000(4bytes))
• random insert/select on 1 million records
• key range is "00000001" to "01000000"(8bytes) and value range is 1 to 1000000(4bytes))
• sequential insert/select on 50 thousands large records
• key range is "00000001" to "01000000"(8bytes) and value range is "000...001" to "100...000"(100kbytes))
• random insert/select on 50 thousands large records
• key range is "00000001" to "01000000"(8bytes) and value range is "000...001" to "100...000"(100kbytes))

	Lux IO (B+-tree)	Berkeley DB (B+-tree)	Tokyo Cabinet (B+-tree)
1 million records (sequential insert) (s)	2.140 (1.048 with bulk-loading)	2.573	1.472
1 million records (sequential select) (s)	1.878	2.208	1.245
1 million records (random insert) (s)	3.587	3.875	5.134
1 million records (random select) (s)	2.424	2.772	3.051
50 thousands large records (sequential insert) (s)	71.357 (non-cluster index)	91.517	83.482
50 thousands large records (sequential select) (s)	79.243 (non-cluster index)	over 10 minutes	62.379
50 thousands large records (random insert) (s)	77.099 (non-cluster index)	- (too slow)	925.255
50 thousands large records (random select) (s)	101.833 (non-cluster index)	- (too slow)	591.589

Lux IO is very fast in both sequential and random access and also can hold large data efficiently like no other.
It gives super good performance for large expanding data as a value. (it's not in the list.)

All the tests for Berkeley DB and Tokyo Cabinet are appropriate tuned to get maximum performance under the same condition as Lux IO as far as I could.

Lux IO is a free software under the terms of the GNU Lesser General Public License.

• luxio-0.2.2.tar.gz (2010/02/03)
• luxio-0.2.1.tar.gz (2009/03/02)
• luxio-0.1.0.tar.gz

Repositories

• https://github.com/feeblefakie/luxio
• http://luxio.svn.sourceforge.net/viewvc/luxio/

It should work in all posix operating systems, but I only tested in Linux and MacOSX.

$ ./configure
$ make                      # nothing to compile for now.
$ sudo make install     # copy header files to the specified directory

sequential insert on 1 million records

#include <luxio/btree.h>
#include <iostream>

#define NUM_RECORDS 1000000

int main(void)
{
  Lux::IO::Btree *bt = new Lux::IO::Btree(Lux::IO::CLUSTER);
  bt->open("test", Lux::IO::DB_CREAT);

  char str[9];
  for (int i = 0; i < NUM_RECORDS; ++i) {
    sprintf(str, "%08d", i);

    Lux::IO::data_t key = {str, strlen(str)};
    Lux::IO::data_t val = {&i, sizeof(int)};
    bt->put(&key, &val); // insert operation
  }
  bt->close();
  delete bt;
  
  return 0;
}

sequential select on 1 million records

#include <luxio/btree.h>
#include <iostream>

#define NUM_RECORDS 1000000

int main(void)
{
  Lux::IO::Btree *bt = new Lux::IO::Btree(Lux::IO::CLUSTER);
  bt->open("test", Lux::IO::DB_RDONLY);

  char str[9];
  for (int i = 0; i < NUM_RECORDS; ++i) {
    sprintf(str, "%08d", i);

    Lux::IO::data_t key = {str, strlen(str)};
    Lux::IO::data_t *val = bt->get(&key); // select operation

    // do something with the val
    // *(int *) val->data
    
    bt->clean_data(val);
  }
  bt->close();
  delete bt;
  
  return 0;
}

compile and excute

$ g++ put_test.cpp -o put_test -O2 -L/usr/local/lib -lluxio -lpthread -I/usr/local/include
$ g++ get_test.cpp -o get_test -O2 -L/usr/local/lib -lluxio -lpthread -I/usr/local/include
$ ./put_test
$ ./get_test

You can see these examples in example directory.
For other cases, please check test directory for now. It contains a lot of test programs and may be useful to understand how to use.

Lux IO is developed by Hiroyuki Yamada (hiroyu at users.sourceforge.net).
Please feel free to contact me for any comments and bug reports.
There is also an open forum. Please ask questions and report bugs in the forum first so that users can share the information.