discussion.txt

 
- resource identifiers should conform to the uri spec...

	http://localhost:8888/player.next
	http://localhost:8888/player/next

	//localhost:8888/player/next

	/player/next

	nerve://192.168.1.180/player/next	# 'nerve' being the raw protocol

	http://192.168.1.180/player/next


- the query function would take a valid uri, parse it, as well as taking arguments

	nerve.query('http://localhost:8888/player/pl_add', 'a string of arguments')
	nerve.query('http://localhost:8888/player/pl_add', [ 'list', 'of', 'arguments' ])
	nerve.query('http://localhost:8888/player/pl_add', { 'dict' : 'arguments' })

	(or should the second argument be only a string or list, and have an optional third argument that's the dict.
	Either way, you also need to pass a source address too somehow...)

	nerve.query(query, args, src=None) # or should src be 'replyTo' or something
	nerve.query('http://localhost:8888/player/pl_add', 'a string of arguments', src='console')
	nerve.query('http://localhost:8888/player/pl_add', 'a string of arguments', src='localhost:8888/console')

- Populating a device tree???
	nerve.add_node('player', 'http://localhost:8888/player')
	nerve.add_node('player', AliasDevice('http://localhost:8888/player')) # or this??
	nerve.add_node('medialib', MediaLib(config='medialib')

	config = {
		'medialib' : { 'path' : [ '/media/media/Music', '/media/media/Torrents' ] }
	}



	nerve.query('player.next', '')
	nerve.query('http://192.168.1.180:8888/player/next', '')
	nerve.query('192.168.1.180:8888:player.next', '')  # ???


-----

2014/07/22:

config consists of a list of 'servers' (instead of portals...) and a list of root devices, and a device type
The device name must be unique, and can then be used as the root index into the global configs.  The
device type could maybe be stored there too...

config = {
	'devices' : [ 'player', 'medialib', 'rgb', 'deskclock' ],

	'player' : {
		'type' : 'vlchttp.VLCHTTP',
		'vlcuser' : 'person',
		'vlcpasswd' : 'somethingencryptedisuppose',
		'vlcport' : 7999
	},

	'medialib' : {
		'type' : 'medialib.MediaLib',
		'path' : [ '/media/media/Music', '/media/media/Torrents' ],
		'updatehours' : 24,
	},

	'rgb' : {
		'type' : 'serial.NerveSerialDevice',
		'file' : '/dev/ttyACM1',
		'baud' : 19200
	},

	'deskclock' : {
		'type' : 'deskclock.DeskClock',
		'file' : '/dev/ttyACM0',
		'baud' : 19200
	}'

	'servers' : [ 'http', 'console', 'udpserver' ]

	'udpserver' : {
		'type' : 'raw.UDPServer',
		'port' : 5959
	}
}



Possibly a new message system?

	msg = {
		'to' : 'player.next',
		'from' : 'udpserver', # <Object ???>
		'args' : [ ]
	}

	- it needs to be able to send a message back to whomever sent it (the 'from' field)
	- it can be partially represented as a uri:
		udp://localhost/player/next
		udp://localhost/player?cmd=next
		udp://localhost/player/add?url=file:///media/media/Torrents/song.mp3
		/player/next	# (localhost, doesn't need protocol)


- the primary methods for functions to be invoked are: web request, raw request, and internally code (including code in pyhtml pages, but
  necessarily including all helper functions??)
  It must be possible to have one function that works for all three

Web Request:
    http://localhost/rgb/ir?P&4004&100BCBD
    http://localhost/rgb/ir?a1=P&a2=4004&a3=100BCBD

Raw Request:
    rgb.ir P 4004 100BCBD
    /rgb/ir P 4004 100BCBD

    /rgb/ir type=P&addr=4004&code=100BCBD	# this doesn't work for the serial RGBNodes, because it's too much to parse out

Internal:
    nerve.request('/rgb/ir', 'P', '4004', '100BCBD')
    return_value?? = nerve.request('/rgb/ir', 'P', '4004', '100BCBD')

    playlist = nerve.request('/player/getplaylist')
    playlist = nerve.request('player', cmd='getplaylist')	# or the command is an argument
    player = nerve.get_device('player'); playlist = player.getplaylist()	# this isn't good because a request should be something specific
    player = nerve.get_device('player'); playlist = player.req_getplaylist()
    playlist = nerve.request('player.getplaylist')
    playlist = nerve.request('player/getplaylist')
    playlist = nerve.request('player getplaylist')

    playlist = nerve.request('player enqueue file:///W:/nerve/config/raven_medialib/playlists/Default.m3u')
    playlist = nerve.request('/player/enqueue file:///W:/nerve/config/raven_medialib/playlists/Default.m3u')
    playlist = nerve.request('/player/enqueue %s' % (url,))
    playlist = nerve.request('/player/enqueue', url)

- how do you reconcile the param names in web requests, and the lack of such param names in raw requests...
  compared to the internal requests, it seems more natural to have non-named params, so they can either be thrown away when you
  pass them to the function or....


	def req_ir(self, type, code, code2=None):
	    if type == 'P':
		send_panasonic(code, code2)
	    elif type == 'S':
		send_sony(code)



def request(*args):
    if len(args) == 0:
	return False
    elif len(args) == 1:
	args = args[0].split(' ')

    ref = nerve.resolve_uri(args[0])	# this would return an object that can be used to send the command to another computer, or
					# which can itself be invoked??? this doesn't work!! you need to treat those things differently
					# still...
    obj, request = ...
    obj.request(request, args)
    # etc

class Device:
    def request(self, req, args):
	func = getattr('req_' + req)
	func(args)

class HTTP:
    def request(self, req, args):
	r = requests.post(req, args)



Device.request(address, request, args)
- for a local request, the address would be None (this might be a lower level method, with the address and request being split)
- either way, this argues for a distinct request param that is neither the address nor the args
- now the question is, is the device name part of the address, or part of the request? ("http://localhost/", "player/next")
  or ("http://localhost/player", "next")
- I prefer the second one, but the main request() function should automatically parse this a single string, or a string + arguments?...  
    "http://localhost:8888/player/next"
    "udp://localhost/player/next"

    "localhost:8888/player/next"
    "/player/next"

    "player/next"	# this is nicer to write, but it's potentially conflicting with the missing-protocol syntax.  However, it could solved
			# if it is possible to include local devices while searching dns records.
    "http://player/next"	# if that were the case then this syntax would also be valid, which... I'm not sure if that's a good thing or not,
				# but I kind of feel like not...  depends on how we parse it...

    "http://localhost:8888/player.next" # kind of a bad idea

- should you abandon the whole . syntax in favor of a uri?
- yes, i like the idea of a unified namespace.  The only place I really used dot syntax where the direct device is not the top level is from the
  phone, for eg. 'cr.player.next', and since cr just refers to a server, it could really instead be udp://cr/player/next where cr is either
  specified in /etc/hosts, or there could be some kind of custom hosts file or host discovery or something
- if this is going to happen, then there should be a unified resource tree, such that no matter what protocol you use, the same path structure
  will apply.  If a request can't be satisfied on a certain protocol, then it will return an error or redirect message

- now... where does this leave the need to specify 'default' devices, such that different components can control devices through a redirectable
  name (ie. local device player redirects to a remote player device)

---------

2015/01/19:

player/load_playlist?arg1=Nine+Inch+Nails
player/load_playlist?a=Nine+Inch+Nails


---------

2015/03/15:

OBJECT FILESYSTEM

internal vfs:
/
    /modules
        /modules/datalog
            /modules/datalog/controllers/DatalogController
            /modules/datalog/devices/DatalogManager
            /modules/datalog/devices/DatalogDevice
        /modules/serial
            /modules/serial/devices/SerialDevice
            /modules/serial/devices/NerveSerialDevice
        /modules/http
            /modules/http/servers/HTTPServer

    /devices
        /devices/datalog (type = /modules/datalog/devices/DatalogManager)
            /devices/datalog/temperature (type = /modules/datalog/devices/DatalogDevice)
        /devices/tv (type = /modules/local/TV)
            { serial = /devices/rgb }
            /devices/tv/power (callable function essentially)
        /devices/lamp/ (type = /modules/base/ConfigObjectTable)
            /devices/lamp/power (type = /modules/base/SymbolicLink) -> /devices/sensors/relay_toggle
            /devices/lamp/is_on (type = /modules/base/CustomMethodOrSomething):
                intensity = nerve.query("/devices/sensors/intensity")
                if intensity > xx:
                    return True
                return False
    /servers
        /servers/http (type = /modules/http/servers/HTTPServer)
            { port = 8888 }



An alternative to /modules is /types.  They can also be grouped by type
/types/servers/http/HTTPServer
/types/devices/datalog/DatalogManager
/types/devices/datalog/DatalogDevice


---------

2015/04/20:

EVENTS

Ok so things are currently set up for querying.  You send a query to a device, and it returns some data as a response, and/or controls
something in response.  The queries come from an originating source, usually from a request received by a server, but also from other
sources like the datalogger, which sends out queries periodically in order to collect the data it needs to log.  It's a one directional
system though.

What I need now is notifications... a device-originating means of getting something else to do something.  Otherwise we'd always have to
poll the devices for changes of state, or (what I'm mostly thinking about atm) IR codes being received, that should be acted upon.  It
wouldn't be feasible to constantly poll the device object to see if an IR code has been received, even if it's handled in the device
object and not the device itself (thus eliminating the issue of sending serial messages over the wire).  It would be better if there were
a way for objects to receive notifications without requiring them to send a query first (for each notification).

So then the question is, what is the best way to implement this:
The receiving object can send a query to the device to register with it (possibly a nerve.Device-generalized query and system thing).
Would it have to register every time the target device is created?  Would it be a runtime pointer or would the device be given basically
a return query to send in response to that notification (if /devices/rgb/ir receives an ir notification, then send a query to
/devices/remote/notify or something like that).  What if the device is deleted and readded?  (It's easy to do in config atm)  Would all
the registrations be lost?

Or you could have a table of some kind that maps notifications to target queries (possibly embedded in the internal vfs:
/notify/devices/rgb/ir would be a list of target queries to call).  This would be independent of the devices, so changes to the devices
(delete and readd) wont eliminate the notification links/callbacks.  They would be set as part of configuration, and would persistently
stay around until they are unlinked through configuration.  When a device wants to send a notification, it would send it to the list
object which would send it to everyone else.  The actual path used to send notifications wouldn't even have to match the device's path;
it could be a somewhat generic name, like '/notify/ir', and no matter what device receives an ir code, it would notify to that path, so
that anyone that wants any ir codes only needs to register in one place.

But what about a simple response like (in response to certain ir code) 'wait for 2sec and then send this query'.  How would this be done
(assuming we will have some kind of rule/routines/triggers/events system)?  If a simple query is sent to notify something, then you'd
have to set up something that can receive that notification query, which can then do that behaviour...

An alternative would be that all notifications cause an 'event'.  So the device would send it's notification to the events system, which
would have some kind of list of events to perform (some time delayed, some triggered by notifications), and then you'd always have an
event/routine run when a notification comes in, and that would then execute any queries, if required.  If a device needed to register
for a notification, it would create it's own event in the event system which would fire when the required notification occurs, which could
then just send a query to the device to handle the rest itself (query is the same as calling a function on the device object).

The events system would also handle situations where you need to check if a condition has been met (you might have to use only polling
i mean, rather than notifications).  Like 'if the temperature falls below a certain amount and it's a certain time of day, then do a
bunch of stuff in response'


I'm starting to lean in favor of a system where 'events' are a separate thing, like devices and servers, and devices can 'notify/signal'?
So there'd be a normal directory called '/events/', and in it would be some kind of organization which I haven't decided on yet.  When a
device received some data, it would call a function somewhere in the event system, which would then run all the event objects that match.
All events (by convention) would be in '/events/', but not all would be activated.

    "events": {
        "irtest": {
            "N:628": {
                "__type__": "events/Event",
                "code": "nerve.query('/devices/player/toggle')"
            },
            "N:690": {
                "__type__": "events/Event",
                "code": "nerve.query('/devices/player/next')"
            },
            "N:650": {
                "__type__": "events/Event",
                "code": "nerve.query('/devices/player/previous')"
            }
        }
    }


    "events": {
        "ir": {
            "irrecv": {
                "toggle": {
                    "__type__": "events/Event",
                    "trigger": "N:628",
                    "code": "nerve.query('/devices/player/toggle')"
                },
                "next": {
                    "__type__": "events/Event",
                    "trigger": "N:690",
                    "code": "nerve.query('/devices/player/next')"
                },
                "previous": {
                    "__type__": "events/Event",
                    "trigger": "N:650",
                    "code": "nerve.query('/devices/player/previous')"
                }
            }
        }
    }

If the second method was used, would the event system have to traverse the whole tree to find events that match the trigger?  Or would it
correspond to the data value of the notification?  In the latter, the device would emit a signal '/events/ir/irrecv', with the data
corresponding to the code received (or the new value of a sensor).  The event dispatch thingamasystem would then call all the events in
'/events/ir/irrecv/*' that have the appropriate trigger (the trigger mechanism could be an argument thing that is automatically handled in
Event.__call__().  The names of the events inside /events/ir/irrecv would be whatever you want it to be.

So now my question is, where does the signal go?  The device has to call a method somewhere.  Should that be a special notification method?
Should it be global (and have it call a method on EventThreadPool?) or should it even just be a regular query?  The argument against a
query, is that there'd have to be special support (in the form of a __call__ method on the directory object) that would then run all the
events in the directory...  oh hmm... thoughts...

There is an ambiguity here in function, in that if you wanted to query all the objects, and it was possible to query a parent directory
and have it query all sub objects, this could be really bad if you *don't* want to query everything but accidentally try to query a
directory.  It wouldn't be hard to have this go through a special function, like nerve.notify(), that will have the query-everything
functionality, but would this be useful to have as a generic thing, that could be done on any queryable object?  What if you used the
* (wildcard) notation though, in a regular query: nerve.query("/events/ir/irrecv/*", "N:690"), and then it would query each object in
/events/ir/irrecv, with the same arguments ("N:690")?  It's not ambiguous with the other functionality, although it would have to be
treated as a special case, I guess, in nerve.query.

That or you just have nerve.notify("/events/ir/irrecv", "N:690"), or it could be named nerve.query_multiple() or something like that...
more generic/ordinary sounding.  What if you still had the wildcard syntax, such that you could specify it in different places

nerve.notify("/events/ir/irrecv/*", "N:690")
nerve.notify("/events/*/irrecv/*", "N:690")

nerve.notify("/devices/player/*/stop")      # stop each individual player object

On a side note, the advantage of having all generic directory objects instead of special 'directories' that have additional methods, is
that it's easy to maintain with filesystem style commands.


---------

A note about queries:

player/load_playlist?name=Nine+Inch+Nails <--- should be what's used...
we should pass a query string to ardunio instead of the space separated query sent now


---------

2015/07/30:

MODULES/PER-MODULE CONFIG/PER-OBJECT CONFIG/

You need a way to start and stop servers
    - should this be relevant of all/most threads?

You need singleton classes of some kind
    - should models be singleton classes, or should they be individually created as needed (like in controllers)


- EventThreadPool needs a way to create the object/thread that runs events in the background.
    - should this be just a thread/task
        - should threads/tasks all be common and represented in the objfs somehow? /tasks/EventThreadPool

    - or should this be 'module specfic', even though events aren't in a module of their own (they could be?)
        - if so, should it be created during an init() call to the module, or should there be an automatic one
          where when the module is loaded, the module namespace is searched for a class of type Module, which
          it then creates and calls init on?
            - I already don't like this idea because not all modules need this
        - if using an init() call on module load, should all modules be loaded, and init called, before the objfs data is initialized?

- medialib updaters needs should be on a per module basis

- we could have a multi-stage initialization process... 1) load modules, 2) populate objfs, 3) start threads/servers

- what about DatalogManager???  Where should that be created and access and everything?  Could it just be a module specific task?




SECURITY & PERMISSIONS:

- you need some kind of access restrictions on objects
- writeable vs read-only: to control whether you can use set_child or set_object
- execute only queries vs execute all non-_leading methods: set so that when using query(), you can only execute approved methods



Servers:
- http
- console/teriminal
- simple query thing
- irc?

- the server should know what kind of requests it can receive, or what kind of data it wants from the controller, and communicate that to the controller


Types of Requests:
- get html data
- get binary files
- query things
- interactive text (terminal)

What ways can the user interact with the system:
- control a conceptual device (player, stereo, ...)
- request and display data (immediate values such a temperature, or historical data such as trends)
- retrieve files (possibly virtual files)

- chat or communicate with others?
- leave messages for other users
- leave reminders for yourself

- repeat events periodically or after a time
    - like a tea timer where you click the button and then it, by some appropriate means, notifies you.
      perhaps a chime on stereo, or chime on phone (the app could make an android notification.  is there a way to do that through web??)


- thinking about the system reminding you about something later... there could be multiple means and the system should have a way of
  knowing what means is appropriate, such as knowing that the stereo is on or off, or knowing that you aren't home and the android
  app wont work, so it should send an email or something instead



- so you have a view compiler.  A controller handles a specific request, and returns a view.  I don't know exactly how is best to do that but
  for now, the controller being is object that returns the final output.
- as it is now, the controller creates a view and passes it the data, but the controller can instead create partial views for the data, and
  send that to a view compiler which assembles the partial views in a way appropriate for the server who's request it is handling (ie.
  there can be views for each part (tabs, menu, content, footer, etc) and then the view compiler could use some or all of those views to
  produce an output.  So the view compiler would tailor the output appropriately for the server, whether it's a console or http)



- you can talk of the controllers as having an api to access the function of that controller.  The other side of the api in most cases is defined in the js and html files
  but you use a common api for accessing shell/pyexec/etc.
- the controllers should strictly handle processing of requests, with possible special requests for features like tabcomplete
- the server, or a piece of code between the server and the controller, should take of the actual user interaction
- in the case of console and tcpserver and stuff, there should be a common piece of code on top of the server that runs the controller... should this code be a subclass of the server??
  so you might subclass Console or TCPServer.  The problem here is that you wouldn't be able to share the code easily between two different superclasses... could be a mixin?




for websockets (as well as html requests), there is a view (phtml + js + css, or View object) that implements the controller's api!
the controller currently defines the view...
a controller could be accessed through multiple different types of servers, so eg. the irc client controller could either be accessed through another irc client
or through websockets, using a custom html view

what if you had settings in the controller objects specifying what type of view to use, so that it can be server specific


there is a fundamental difference between servers that result in a single request and response and servers which continually send and receive requests and responses,
possibly out of order (like a response, sending back data, is not always triggered by a request, in the case of the shell printing the welcome message and stuff...)
there is something that isn't *exactly* a server, it's a server thread that handles a single connection.


-----------------------------------

2015/10/25:

Ownership and Permissions of Objects

- possible access checks
    - read/access an object
    - modify an object
    - read a child of an object
    - set a child of an object
    - delete a child of an object


owner, group? role??

r readable
w writeable (add/modify?)
x queriable??


- we want to make some nodes undeletable, but still allow you to add child nodes.  Making the parent uneditable doesn't work either
  because we just want some nodes to be fixed (namely nodes that are automatically created, like event groups and mysensors nodes)



-------------------------------------------------

2015/11/09:

player/load_playlist?$1=Nine+Inch+Nails

$1, $2, etc

or, which will be parsed exactly the same
player/load_playlist?%241=Nine+Inch+Nails


---------------------------------------------------

2016/06/11:

I'm considering re-introducing something like nerve.notify, or nerve.emit, or nerve.trigger, which will be different than the one before.
The important difference it that it would contain the *originating* ref instead of the target ref, so:

nerve.notify("/devices/rgb/ir", "N:690", type='change')

This might even use the same notifications module I recently added, in which case user notifications would be identical in origin to
all signals of conditions.  I'm debating whether these should be called notifications, signals, events, messages, or something else.  These
would be more similar in function to signals, or events in terms of html/dom.

Other things could then subscribe, which would run some code, call a callback, execute a query, or something when an event/signal/notification
matches certain criteria.

The type='change' part would be to indicate it's a notification that the value of /devices/rgb/ir has changed or updated.  In the case of a
notification to the user, the type might be 'message' or something else.

nerve.subscribe(src="/devices/*/ir", type='change', action="/events/serial/ir/*")

So the above would add a trigger subscription which, when a notice comes in with type change, and with a src ref that matches the regex, it will
query the given target.  If action was a function, it would be called.

This could even be added to the /devices/notify object, such that /devices/notify/subscribe?type=change&... would add a subscription to a list
that's saved; either a config setting or as a child to the object somehow...


It could instead be like with dom, where an event/signal would bubble up from the originating object, via a special function on ObjectNode.

self.trigger('change', 'ir', "N:690")

and it would check for its own listeners, and pass up towards its parent, building the reference as it goes up.  It might want to continue bubbling,
or maybe a special option in the listener could stop the propogation if it wanted?

self.parent.trigger('change', 'rgb/ir', "N:690") ???
(or would it be better to always have an absolute refpath.  nerve.query and stuff couldn't handle a relative path like this...)


In the first method, everything would call the same static method/global object, which would process and dispatch everything based on a global list of
listeners.  (They could be organized into subdirs and stuff if needed)...  In the second method each node in the network has listeners distributed throughout
the object tree.  It would be harder to manage all the listeners globally, but it would be easier to have specific listeners for simple things, and global
listeners for more generic things.  (Like if an object wanted to catch its own event, or one from just above or just below that object, it can do so easily
without cluttering a global name space, and user-configured triggers would attach to the root of the tree).


keep in mind this is also an issue for IRC stuff.  The irc device needs to send messages/notify/signals when an irc message or something comes through.
Right now, it has a local directory for triggers that it will query* when an event happens.  It would be nice to have both kinda, a local trigger and a
global trigger for the same event