Discussion:
kind of ot: the concept of software integration
(too old to reply)
cr88192
2004-01-29 04:09:39 UTC
Permalink
I will try to approach this from a general standpoint.
I request people's oppinions on this subject as well.
I know this is boring, and the talk of too damn many hypists (can I claim to
be any different: I don't know...).

Protocol

first of all, general requirements I guess:
the protocol should be fairly simple (a partial but working implementation
should be possible quickly, and other implementations may not be able to
expect "correct" behavior in all cases).

that it be possible to easily add whatever features are desired to the
protocol without possibly breaking capability, and being capable of easily
determining what is supported by a peer (it should be possible to readily
mix-match features from different extensions in the same message as well).

there should not be schemas or requirement of statically defined information
about what is being connected to or how it should operate.

that more or less any node can export functions and give references to
objects (and not to have, eg, a global ip address, a registered hostname,
...), and there is little assumption about the reliability/longevity of
references.

that the protocol be capable of handling dynamic typing effectively, or at
least not treat it as a "fringe" feature.

the typesystem and message types should be easily extended by possible
ad-hoc extensions.

no real assumptions should be made about the typesystem/object
system/langauge semantics/... in the base protocol (though, this should be
competently addressable by extensions).


these are only the basics as I imagine them, but could be a starting point.
I imagine more advanced behavior can be delayed. I will state that
performance is not a major concern.

I will argue that many existing systems have various deficits:

soap (rpc):
seems complicated;
would likely require a fairly developed xml parser, as opposed to the "half
assed" parser of many apps, often maybe only supporting tags, attributes,
and bits of text;
does not seem that it would work very well with an only partially complete
implementation;
does not seem straightforwardly extensible (or at least not in the "keep
basic stuff, pile more ad-hoc crap on top" type mentality, eg, it seems to
view itself too much as the "end all" protocol).

xml-rpc (rpc):
in itself lacks a good extensibility mechanism;
no-one seems to be behing spec'ing and archiving various extensions (eg:
like what is done by the jabber group);
existing people dealing with xml-rpc have virtually o interest in anything I
say (well, not like I say anything woth while);
otherwise I am a strong supporter of xml-rpc.

more conventional rpc's in my mind don't really need to be listed
(personally, I don't consider them really anywhere near sufficient...). I
know, this could be classified as trolling, but I will that this as my
oppinion.

others?


http (transport):
it requires incomming connections;
little facility exists for effectively merging/passing messages between
multiple nodes (another protocol would probably be needed for this, likely
tied to the specifics of the server).
with http, little info is retained about the caller (or at least not without
the info being stored in the payload, or special addon features to http);
for the case where things can run on a server and these aren't a problem it
may well be a decent option (except I am not as much a fan of requiring
multiple connections for concurrent calls).

email (transport):
there is some variation in the workings of the email system (eg: smtp/pop
vs. hotmail), possibly some kludging would be needed to seperate user
emails, eg, from rpc calls. I am somewhat unsure about performance, and
would think some additional security measures might be needed.

xmpp (transport):
I am a personally a fan of xmpp. it doesn't seem to have many faults in my
mind (beyond the lesser concern of performance). specs for transport of both
xml-rpc and soap over xmpp exist. even without them: adding support is
fairly straightforward, and the handling of unknown extensions is
straightforward (they are ignored).

there may be other worthwhile transports I dont know about. others?

protocols I believe are more a matter of politics than of technology though,
which is why I believe a strong voice is needed (though, there is allways a
large chance that an inferior technology will win...).
I do not believe there exists a "one true protocol"...


Language

well, this is language specific land.
preferably the interface should be fairly straightforward or transparent. I
believe it should be possible to retain as much of the language's native
features/quirks as possible for supporting nodes (this is where
extensibility goals come in), but in case less is supported things should
remain operable (eg: special types are sent as more general ones, or maybe
the message fails if the other end does not know of some "required"
feature...).

for more primitive languages (eg: c). it may be useful to just statically
define some amount of the interface and use a tool to spit out a glob of
code to mask over calling issues (eg: making them look like local c
functions). protocol issues should still remain a minimal concern of the
code, and it should still be capable of dealing with other languages with
different behavior (eg: recieving a call with invalid arguments for the
language/function results in a fault being returned...).

(in fact, I would be quite supprised if say, a haskell, smalltalk, and lisp
node could communicate without making a large number of comprimises wrt
interface).
thus, I will assert that external interfaces should be "as minimalist as is
reasonable".

languages is probably the main place things get ugly, but I believe the
language/application issues are still important at the protocol layer...


I still believe this is only the basics. more complex issues, eg, finding
stuff of interest, moving "complex" data/types, copying code,
spawning/joining remote threads, transactions, ... I believe are outside the
present scope of this ideal (more so, I believe these kind of things should
be built "on top" of the rpc protocol, as opposed to being part of it).
necissarily, using features of this sort hinder compatibility (and I believe
this is the "correct" behavior as well), but I believe it is very important
that they remain usable.


ok, I am probably stupid.

--
cr88192 at hotmail dot com
http://sourceforge.net/projects/bgb-sys/
http://bgb-sys.sourceforge.net/
cr88192
2004-01-30 03:14:57 UTC
Permalink
[repost from a recent email, less interesting...]
------
this was not the protocol spec (rather, an idea reasoning about it).

actually, the lower few layers are based on existing protocols:

XMPP/Jabber (this is what I am using as the transport):
http://www.jabber.org/ietf/draft-ietf-xmpp-core-22.html
http://www.jabber.org/jeps/jep-0078.html

The spec for the transport of XML-RPC over Jabber, coined "JabberRPC" by
those who wrote the spec (I am using this, my protocol intends to reatin
compatibility with xml-rpc).
http://www.jabber.org/jeps/jep-0009.html

XML-RPC:
http://www.xmlrpc.com/spec

ok, so where do I come in? I designed a basic extension feature:
http://bgb-sys.sourceforge.net/2004-01-18_XMLNS_EXT_XMLRPC.html

I plan to use this as the basis for the next few layers up.

status: basically, what I have spe'ed thus far I have implemented.
I know, this would be daunting to implement and will get fairly bad
performance...
(hmm, thought, an alternate protocol intended to get better performance).

I am viewing changes as necessary (eg: link will likely be replaced by a
more capable construct):

<reference url=... id=... type=.../>
contains no data, but has attributes.
url=url of node containing object
id=node specific object id
type=semantic type name of object (rough, ignorable)
default: unspecified, may just recieve messages.
closure: object is a closure.
object: object is an object of some sort.
process: object is a process.
others may be added, any that are unknown default to 'default'.

<reference url="cr88192 @ jabber.org/lbxrpc-5G+a" id="#0205a6fc"
type="process"/>

maybe a message type:
<call id=...>
... (otherwise it looks like an xml-rpc call).


ok, nevermind.
I am feeling stupid.

------
now I am thinking:
screw it, I will probably re-implement something similar to my mrp
protocol...
at least then I wont have to deal with the complexities of a long and
complex path between my language and net code (currently having to be
converted multiple times, having to work around ugly typesystem boundaries,
...).

screw compatibility or working effectively over the internet.
mrp2 will be:
specialized to my language (like mrp);
designed to be fast and fairly cheap;
based on a stack machine (though this time I will likely seperate messages
and have something resembling version codes...);
...

most notably, I will skip out on all the crap I have to deal with that is my
net/rpc/objtype/netparse/... systems...
all my efforts are taking too damn long, and are too much work, and getting
little gain.

no one gives a crap what protocols I use anyways, jabber-rpc will be used
for "portability"...

...

or something...
all I do is probably worthless anyways...

Loading...