Stroppytux

The Node database is a database concept I am working on which removes the restraints of schema design.

A basic view of the concept is to simply remove any concept a hierarchy. Each piece of information is assigned a node number and a list of named references. These references are then used to bind information together in a neural type relationship.

Currently I am writing the database in Python due to it being such a lovely language and due to the ability to convert parts of the code into c in the future using ctypes.

So far I have the system is working as a basic storage and reference system. I still have to impliment the security system, change the node reference system from a serialized object into something else (probably innodb storage for the node references), and make some changes to the interface language itself.

If you would like to help me by using the the node database, or have a look through my code, the code can be checked out anonymously through my svn server at: http://www.stroppytux.net/svn/node/trunk or you can browse the source online at http://www.stroppytux.net/wsvn/node. I would greatly appreciate any information or ideas on your experience, and would be happy to give write access to the repository to anyone who would like to contribute. (send an email to me at gary at “my domain name”)

How the data is stored and retrieved

When creating a node (eg. add [[['color','red'],[1:['colors']],’#FF0000′]]), the data segment is assigned a node number and stored on the filesystem under the node number. We then create a node object that contains only the meta information and linking information. The reason we seporate the data segment from the node object itself is so deep queries (eg. get links links data [1:['hex']]) can be performed extremely quickly. Unlike in relational databases, we can traverse extremely deep into our information structure with little overhead, then only when we get to the actual information we want, we then only read and return that information. (Think using 100 joins to get to some obscurely related data without reading every tables data as we go along)

The security system

The security system (still to be implemented) will be implemented using the node based object system itself. Node ZERO is reserved as a root security node. When defining a node of data, a link to your current security node will be created with a security bit set. To extend the security model, you will simply create new security node that links to a higher security node. This lets users implement there own security structure and type. (eg. creating a new group of users will be as simple as creating a security node and placing nodes into this node.) We will then also be able to use certificates (simply by storing the cert into the data segment), or any other means of security. The security node will contain the rights assigned to all linked nodes and will traverse. Multiple security nodes can be attached to each node.

The language

The language is currently extremely minimal. The goal is to keep the base language to under 10 commands. (currently: get, set, add, remove, filter, start, undo, commit) We then have a set of meta’s that the actions work on (currently: link, data, mime, meta, created, modified, accessed), and a special word ‘as’.

Interfaces

Interfaces will allow you to simplify retrieving common structures of data. Once a user defines an interface, calling data through the interface will return any data proccessed by the interface. For instance, calling the red node through the colour interface could return the hex value, cmyk value, names in all the languages, and any objects that are red.

The ‘as’ statement

One of the problems I have come across so far is trying to remember node numbers. When adding nodes to the database, adding an “as [definition]” to the statement lets you assign a memorable name to a node. These names can be assigned to the connection from the client, or to a security node. Assigning the name to the connection itself is volatile and will be removed when the client connection is closed. When assigned to a security node, all users that use that node will have access to the namespace, and so, will be able to call nodes by assigned names. To apply a “global” name, the name can be assigned to the root security node (node 0) if the user has the rights to do so.

Example use

The examples below are used by me to check I havn’t broken anything. If you would like, I can create a proper tutorial. Just send me an email.

# Add nodes to the system
add [[['colors'],[],’Colors’]]
add [[['color','red'],[1:['colors']],’#FF0000′]]
add [[['color','green'],[1:['colors']],’#00FF00′]]
add [[['color','blue'],[1:['colors']],’#0000FF’]]
add [[['color','white'],[1:['colors']],’#FFFFFF’],[['color','black'],[1:['colors']],’#000000′]]
add [[['color','pink'],[1:['colors']],’#FFAAAA’]]

# Add links to a node
add links [1:[2:['red']]]
add links [1:[3:['green']]]
add links [1:[4:['blue']]]
add links [1:[7:['pink']]]
add links [1:[5:['white'],6:['black','blank','clear']]]

# Remove links
remove links [1:[7:['pink']]]

# Remove nodes (And its links in other nodes)
remove [7]

# Get information and data for nodes
get links [1]
get links [2,3,4]
get links [1:['red','colors']]
get links[1:['colors']]

get data [2]
get created [2,3,4]
get modified [2,3,4]
get mime [2,3,4]
get size [2,3,4]
get meta [4,5]

# Adding more information to nodes
add meta [1:['lists']]
add meta [2:['hex','rgb']]
add meta [3:['hex','rgb'],4:['hex','rgb'],5:['hex','rgb'],6:['hex','rgb']]
add links [1:[2:['hex','rgb'],3:['hex','rgb'],4:['hex','rgb'],5:['hex','rgb'],6:['hex','rgb']]]
add [[['color','white','cmyk'],[1:['colors','cmyk','white']],’0,0,0,0′]]
add [[['color','black','cmyk'],[1:['colors','cmyk','black']],’0,0,0,100′]]

# Mixing links to get specific data
get links [1:['cmyk','black']]
get links [1:['rgb']]
get links [1:['white']]
get links [1:['white','hex']]

# Getting specific data
get links data [1:['white']]
get links data [1:['cmyk','white']]
get links data [1:['hex','white']]
get links modified [1:['hex']]
get links links [1:['white']]

### Imitating relational data (ALTHOUGH YOU WOULDNT WANT TO DO THIS)
# Pink has already been removed, so 7th value will be false
remove [1,2,3,4,5,6,7,8,9]

# Create a colors table
add [[['table','colors'],[],’Colors’]]

# Add a red row to the table
add [[['row','1'],[10:['row','1']],’1′]]
add [[['name'],[11:['row','1','name']],’red’]]
add [[['hex'],[11:['row','1','hex']],’#FF0000′]]
add [[['cmyk'],[11:['row','1','cmyk']],’0,100,100,0′]]

# Add a green row to the table
add [[['row','2'],[10:['row','2']],’2′]]
add [[['name'],[15:['row','2','name']],’green’]]
add [[['hex'],[15:['row','2','hex']],’#00FF00′]]
add [[['cmyk'],[15:['row','2','cmyk']],’100,0,100,0′]]

# Add a blue row to the table
add [[['row','3'],[10:['row','3']],’3′]]
add [[['name'],[19:['row','3','name']],’blue’],[['hex'],[19:['row','3','hex']],’#0000FF’]]
add [[['cmyk'],[19:['row','3','cmyk']],’100,100,0,0′]]

# Get a list of rows (SELECT `c_id` FROM `Colors`;)
get links data [10]
# Get the values of row 1 (SELECT * FROM `Colors` WHERE `c_id`=’1′;)
get links links data [10:['1']]
# Get the value in the name column from row 1 (SELECT `name` FROM `Colors` WHERE `c_id`=’1′;)
get links links data [10:['1']] ['name']