Email Spigot hacks and questions

[thehelt]

Hi,

i hacked a little plugin to enable loading a mailbox and visualizing it as a dynamic graph. I copied lots of the email spigot importer, but now i run into some problems as i load the dataset (500'000 emails) => at least 500'000 DynamicDouble entries in total.

My simple approach iterates over all emails. For each mail sent from A to B i add an entry into a sortedset(for <A,B>) with the sent-date. After iterating over all emails, i have complete "histories" for each edge and are able to contruct DynamicDouble as done in the dynamic graph generator.

It works for small datasets but it is neither elegant nor efficient.


How do i retrieve the EdgeDraft/Edge (which has been added to the container already) and then retrieve its edge-attribute (the dynamic one), manipulate it and write it to the container again?

Is there a possibilty to get a the weight of an edge at a certain point in time?
Furthermore, is there a possibility to set dynamicDoubles adaptively? To be more precise: i'd like to have a (unsorted) list of events (no interval, just a point in time) iteratively inserted/added to the "lifeline" of an edge. When callingedge.getWeight(pointInTime) id like the return value to be the amount of all events on the lifeline before pointInTime. Is that already implemented somewhere in the dark depths of the code or is that up to me?

[mbastian]

My simple approach iterates over all emails. For each mail sent from A to B i add an entry into a sortedset(for <A,B>) with the sent-date. After iterating over all emails, i have complete "histories" for each edge and are able to contruct DynamicDouble as done in the dynamic graph generator.

That make sense. I think it's a valid approach.

How do i retrieve the EdgeDraft/Edge (which has been added to the container already) and then retrieve its edge-attribute (the dynamic one), manipulate it and write it to the container again?

The ContainerLoader has a getEdge() method so you can retrieve the edge (either with it's id if you have one of with the src and dest draft (which you can get with getNode()). Once you have the EdgeDraft you can't get the data as the interface only have setters. The trick is to cast this to a EdgeDraftGetter. In the backend it's the same object, and that will allow you to retrieve the data you need. After, you don't need to add the object again to the container.

Is there a possibilty to get a the weight of an edge at a certain point in time?

Yes. We the weight is a DynamicFloat (instead of a float) you can use the getWeight(low, high) method in Edge. Low and high are basically the interval of time. If you want a point in time, one of the bound can be + or - infinite. That how it works.

Furthermore, is there a possibility to set dynamicDoubles adaptively? To be more precise: i'd like to have a (unsorted) list of events (no interval, just a point in time) iteratively inserted/added to the "lifeline" of an edge. When callingedge.getWeight(pointInTime) id like the return value to be the amount of all events on the lifeline before pointInTime. Is that already implemented somewhere in the dark depths of the code or is that up to me?

I'm not sure I get what you mean by "amount of all events on the lifeline before pointInTime". Sorry. Can you please reformulate?

[thehelt]

Furthermore, is there a possibility to set dynamicDoubles adaptively? To be more precise: i'd like to have a (unsorted) list of events (no interval, just a point in time) iteratively inserted/added to the "lifeline" of an edge. When callingedge.getWeight(pointInTime) id like the return value to be the amount of all events on the lifeline before pointInTime. Is that already implemented somewhere in the dark depths of the code or is that up to me?

I'm not sure I get what you mean by "amount of all events on the lifeline before pointInTime". Sorry. Can you please reformulate?

What I am looking for is the dynamic evaluation (on demand) of an dynamic attribute at a certain point in time. I simply want a list of event time points attached to an edge. And if i call the the e.getWeight(t) for a time t, i want the number of events that lie in the past.

And what i ultimately hoped for was some kind of dynamic evaluation of attributes (some kind of dynamic evaluation (e.g. on demand) of a dynamic attribute).
I have some use cases for the dynamic evalueation:
- for an edge: dynamic edge width: eg. edge_width := Math.abs(source_vertex_size - target_vertex_size)
- for an vertex: dynamic vertex size: vertex_size := Sum of all incoming edge weights - sum of all outgoing edge weights.
- etc. pp.

I think, I will implement a less generic behaviour for the emails (which works on DynamicDoubles).
to add an event ev at time t:
take the dynamic attribute of the edge, split any Interval in it that occupys t, and change subsequently all interval values that are > t. This is a lot of static computation thats happening there (given the fact, that some edges will have DynamicDoubles with hundreds of intervals).

Anyways, thats how I think to implement that.

And i further hope for a "play" button in the timeline.

Kind regards,

helt

PS: Shall i post this stuff on the mailing list rather than in this forum?