A format for Graph Streaming

All questions about the GEXF (see http://gexf.net before)
User avatar
mbastian
Gephi Architect
Posts: 728
Joined: 10 Dec 2009 10:11
Location: San Francisco, CA

Re: A format for Graph Streaming

Post by mbastian » 23 Jun 2010 22:35

Good point, thanks

We were discussing today with Andre and decided we will start with a JSON prototype, as our primary objective is portability. However as it was said the serialization technology is not the real problem, and other library like protobuf could be used and compatible, as far as the Java Objects which are serialized don't change.

So we have to find a serialization library able to serialize in JSON, some pointers?

tcc
Posts: 1
Joined: 01 Jul 2010 23:04

Re: A format for Graph Streaming

Post by tcc » 01 Jul 2010 23:28

have you looked at BSON, http://bsonspec.org/, it's a binary version of JSON.
It's used as the format for mongodb and there are existing java, ruby, c++ libraries for de/serializing

panisson
Gephi Plugin Developer
Posts: 19
Joined: 08 Apr 2010 18:15
Location: Turin
Contact:

Re: A format for Graph Streaming

Post by panisson » 12 Jul 2010 16:53

Hi all!

As Mathieu already pointed, in the last days I started with a simple JSON prototype. For the java serialization I'm using the implementation available at json.org (http://www.json.org/java/index.html).

As I said, this is a preliminary implementation, and is completely open to changes. It is just to give us some idea of how the events would appear in the JSON format. The description of the current implementation is available at http://wiki.gephi.org/index.php/Specifi ... ing_Format. I listed also some open problems and some ways to face it, and I would like to have your opinions about it:
- how to add support to filters
- how to support identifiers to events
- and other improvements (array-type attributes, composite attributes, graph attributes, etc. etc...)
I'll wait for your suggestions.

User avatar
eduramiba
Gephi Code Manager
Posts: 976
Joined: 22 Mar 2010 15:30
Location: Madrid, Spain

Re: A format for Graph Streaming

Post by eduramiba » 12 Jul 2010 21:58

Hi!

For the event identifiers, I don't know if it is possible, but you could use the normal format and, when the events have an identifier, then they must be in the same HTTP request, just before the event. Maybe something like

Code: Select all

{"event-id":"xyz"}
{"an":{"A":{"label":"Streaming Node A","size":2}}}
And as I see a format for complex events as applying changes using filters can be really difficult while not providing so much traffic or performance difference. So I think you could create special events for the most common of these events that delete multiple nodes based on their id range, for example. But for other multiple events, since I guess normally the JSON is going to be written by another program, sending a lot of events is no problem.

panisson
Gephi Plugin Developer
Posts: 19
Joined: 08 Apr 2010 18:15
Location: Turin
Contact:

Re: A format for Graph Streaming

Post by panisson » 13 Jul 2010 12:10

For event identifiers you are right, I would just put it in the same JSON object:

Code: Select all

{
    "id": "xyz",
    "an": {
        "A": {
            "label": "Streaming Node A",
            "size": 2
        }
    }
}
In this way the event identifier cannot be confused with node/edge identifiers, and the absence of the "id" attribute should not be a problem when identifiers are not required.

For filters it is a bit more difficult, and I think you are right in the fact we could create special predefined filters for the most common filtering operations. As an example, the following Json object should change all nodes with id 1-100 to a size 3:

Code: Select all

{
    "id": "xyz",
    "cn": {
        "filter": {
            "range-id": {
                "start-id": 1,
                "end-id": 100 
            } 
        },
        "attributes": {
            "size": 3 
        } 
    } 
}
The problem with this format is that someone could confuse "filter" and "attributes" with object identifiers, and try to change nodes with these ids.
Other suggestions?

Post Reply