HomeCloud

Monads, shmonads and functional programming

noreply@blogger.com (Razvan) — Mon, 19 Nov 2012 18:35:00 +0000

I will be necessarily somewhat long, somewhat wrong and mostly fuzzy, given my own level of understanding, but I am not one to shy away from making a fool of oneself, in the quest for knowledge.

Monads (and related concepts from category theory, like Functor, Applicative etc) are the next level of abstraction in Functional Programming, beyond functions.

Functions

Functions f : A => B allow you to think and program a certain way. Once you get used to this functional programming, you can move on to higher types, higher functions and from there to monads.

A function is basically a transformation: it turns an A into a B. In functional programming, one can take it and do stuff with it.

Functors

Functors F[A] lift a simple function to something more - the signature tells you their secret:

map[B] (f : A => B) : F[A] => F[B]

... they essentially gobble up your function f and return something higher level - pause and reflect on this for a second.

This map or fmap, you could think of it as taking a simple unix command like "wc -l" and apply it to something, as in "find *.txt | wc -l"

In scala, you are used to a slightly different version of the same idea:

List(1,2,3) map (_ + 4)

Why are they useful? Many reasons

For instance – you are normally ‘forbidden’ from using state between two calls to f – you should know that by now. However, inside a functor you could, for the duration of the entire transformation F[A] => F[B]… you could have some state there… this is the beauty of an internalized iterator versus the one you’re used to. That state would be well encapsulated there in that transformation, so it could be used.

How that transformation is executed, is up to the specific functor you use. Some can optimize it, some can be dumb while some can use state (like cache a DB connection between calls or whatever).

Let’s have a quick random example: I could have my own functor, working on lists, which keeps the elements sorted. If I apply a random function to it, the result has to be also sorted. You can see the problem? my functor will keep it sorted while an externalized loop may or may not keep it sorted, depending on the programmer.

MySortedList (1,2,3) map (rand(_))will always be sorted, while List (1,2,3) map (rand(_)) is not…

Without functions and functors, there is no way you could express and enforce that idea – I don’t think…

Monads

Monads go even further. Monads have certain laws which give them certain properties which are very useful once you get used to thinking in those terms.

Monads use flatMap rather than map, with a signature

flatMap (f : A => M[B]) : M[A] => M[B]

As you can see, they also return a transformation, but a yet higher level one, which includes flattening. But because it includes flattening, it can do it in whichever way it wants.

Why monads are more useful than functors – look at the gobble-able: it’s an f : A => M[B] rather than f : A => B – the functors limit you to the shape of the functor, sort of speak – basically if you start with a list of ID’s you will end up with a list of Johns of the same size (or more or less, if the functor is cheating).

Monads are one better, you can start with a list of 5 student ID’s and end up with either 45 grades in a school year or 2 missing registrations… yes, f : A => B has to return exactly one and the same B for an A, while an f : A => M[B] could return for instance empty (called unit) he he…

Keep reading

There’s a lot more to it, as others are trying to convey – try to read as much as you can – there’s no one angle that makes it easy to jump to monad abstractions…

If you are confused by the signatures I used above, it’s ok – no, you don’t have to learn Haskell – there’s an entire series of blog posts to explain the gap...

Note that the random sample above is not kosher, since rand() is not a pure function: it never returns the same B for a given A... but it makes a good point.

OMG, scala is a complex language!

noreply@blogger.com (Razvan) — Sat, 29 Sep 2012 13:16:00 +0000

This was originally posted in Oct 2011 on the blog.coolscala.com

I keep seeing this and, maybe it’s true. Let’s chase this complexity for a bit and go through some of the biggest scala differentiators (from Java or C++, as major OO languages).

Smart compiler infers types

The compiler can infer the types for the most part, so people have to type a lot less repetitive information, which they used to type in both Java and C++. For instance, since “john” is obviously a String, the type of the variable is inferred by the compiler to be String so I don’t have to type it again:

val someone = “John”

I have seen this feature both praised and held against the language as “added complexity”, so I don’t know what to say. I just love typing less and feeling less stupid, every time I declare a value or variable.

Simplified class definitions

Since OO is all about defining classes, scala made do with a bunch of stuff in one go, so that my domain models are dead-simple:

class Person (val firstName:String, val lastName:String)

This, in Java and C++ takes about one page of code: with constructors, getters, setters etc. Scala observed that people don’t need to type one page to inform a stupid compiler that they want a person with a first and last names, so it’s all condensed in this one line, much like a table would look in SQL.

Is this added complexity? Well, I do need to worry about overriding the generated getters/setters ONLY if I need to, so I don’t really know if it’s more complex.

Me? I love this particular feature so much, that honestly, I don’t care what you think J

Unifying methods and operators

All programming languages I know discriminate between methods with names like “append” and operators like “+=”. Some do not even allow re-definition of some hardcoded operators (Java) while some allow infix notation only for operators (C++).

Scala simply makes do with ALL these restrictions and states that the name of a method can be pretty much anything and all can use the infix notation, so I can have:

ListBuffer(1,2,3) append 4

As well as

ListBuffer(1,2,3) += 4

The only difference would be the precedence rules, which are customary in all languages.

Some people obviously would see this as “more complex” than both Java and C++ since they can now do whatever they can… but I see it as “simpler” than both. Operators have been held against C++ before so it really is not surprising that they are held against scala as well.

This is, after all, what makes scala such as perfect DSL framework, allowing natural language such as:

“Mary” with “a purse” and “red shoes” should look “nice”

Types – variance

In both Java and C++, the generics have certain hard-coded and limited behavior (i.e. non-variance) and allow only a few constructs (like List<T extends Person>).

In scala, there is a default behavior, where List[Person] is non-variant, but everything is customizable. If you want co-variance, just tell the compiler List[+Person] or List[-Person] for contra-variance. Just like Java, I can use List[T <: Person] but I can use the reverse just as well: List [T >: Person].

Since scala supports implicits (with finer control than C++), another construct is available: List[T <% Person].

Is this more complicated? Well, just like the operators – it lifts certain limitations of other languages, so it’s both more complicated, since there’s more stuff to learn and simpler, since there’s less rules to live by.

I personally enjoy the extra control… do I actually use it? Not on a daily basis, the defaults are good enough.

Constructors only?

Most languages only allow data types (objects) to be constructed. This is normal in Java and C++.

Well, there’s the flipside, where I can de-construct an object and I don’t mean de-allocating its memory. Consider this:

someone match {

case Person(first,last) => println (“ name is “ + first + ” “ + last)

}

You can see what I mean by de-constructing: took an already created object, someone, and de-constructed into its components. I know this looks foreign to most OO personnel, but trust you me, it is insanely cool and useful. Think what you would have to type in either Java or C++ to achieve the same thing, with if (instanceof) and then type cast and assign two variables and whatnot.

Is this more complex? Well, this is totally new functionality so I guess it is. But I love having it! Trust me, you will, too!

By the way, the match/case construct is way more powerful than your regular switch/case which can only handle constants… we can de-construct types, match constants, match types… and more! Check this out:

someone match {

case Person(first,last) => println (“ name is “ + first + ” “ + last)

case “John” |”Mary” => println (“hello, “ + someone)

case s:String => println (“name is “ + s)

case _ => println (“don’t know what this is…”)

}

Is this more complex? I don’t know… in Java or C++ this is between one and two pages of code. This looks simpler and more intuitive to me… granted, I got used to it but so can you!

Conclusion

There’s more areas of the language, but these are some of the major differences I have time for right now. If you have others, post up and I’ll get into those as well.

I did not get into the functional areas of the language, since that would require comparing with other functional languages and I’m not an FP guy. C++ comes close by allowing passing pointers to methods to other functions while Java 8 I think has some proposed lambda syntax.

Is it more complex? Well, there’s two ways to look at it:

Yes,

There are more symbols and features that one can use
There’s more computer science I need to learn (contra-variance, pattern matching, lambdas, closures etc)

No,

The real-world problems to solve are the same and
To do the same in either Java or C++ is either impossible or takes many times more code… and uglier code at that

What do I think? I don’t really care. To me it was cool to learn these concepts that I had forgotten since university and my new vocabulary allows me to solve the usual problems in just a few lines of code and head for an early lunch, while my mates are still writing some getter or setter…

What do you think?

P.S. This is a great detaiked discussion of complex vs complicated: http://lamp.epfl.ch/~odersky/blogs/isscalacomplex.html

Quick scala – step 1: setup

noreply@blogger.com (Razvan) — Sat, 29 Sep 2012 13:15:00 +0000

This was originally posted in Oct 2011, on the blog.coolscala.com which I'm removing.

If you just want to play with scala quickly – you can use one of the few online interpreters, like http://www.simplyscala.com/or http://www.tryscala.org - these let you run small bits of code interactively. Otherwise, continue reading, to complete a local setup for scala development. You need these:

1.Java

2.Scala (command line)

3.Eclipse with the scala plugin

Installing Java:

1.Download and install a 1.6 JDK from http://www.oracle.com/technetwork/java/javase/downloads/index.html . Do not use a 1.7 for this exercise and make sure you install the JDK not just the JRE. The exact link I use is: http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u27-download-440405.html

a.I recommend you use a 64 bit version, so you can allocate more memory to it

2.Run and install somewhere, let’s say C:\bin\jdk1.6.0_27\

a.I found it a good practice to keep my development tools in c:/bin

b.Make sure to check the “change destination folder” checkbox before letting the installer run…

3.Make sure the path to java.exe is in the PATH

a.Add C:\bin\jdk1.6.0_27\bin to the PATH system variable

b.Define a variable called JAVA_HOME with the value “C:\bin\jdk1.6.0_27\”.

4.Test it by opening a command line window and typing “java”.

Installing scala command line:

1.Download a distribution from http://www.scala-lang.org/downloads

2.Unpack it in a folder, let’s say C:/bin/scala

3.Make sure the path to scala.bat is in your PATH

a.Add C:/bin/scala/bin to the PATH system variable

4.Test it by opening a command line window and typing “scala”. You should get the scala interpreter prompt and then type“exit” to exit the interpreter.

Installing Eclipse:

1.Download Eclipse Classic from the download site: http://www.eclipse.org/downloads/packages/release/indigo/r

2.Install it (unzip it) in C:/bin/eclipsescala

3.To make sure it uses the JDK you just installed (if you went that route) modify your eclipsescala/eclipse.ini and add this line:

a.-vm
C:\bin\jdk1.6.0_27\bin

4.Start it up (eclipsescala/bin/eclipse.exe)

5.Install the scala plugin

a.Go to Help/Install new software

b.Click on Add

c.Type in name “scala291” and the url: http://download.scala-ide.org/releases-29/2.0.0-beta

d.You can see if there is a newer version at http://www.scala-ide.org/

e.Select this source in the drop-down

f.Click “Select All” to select the three features

g.Go ahead with the installation

6.Restart Eclipse

7.Read the detailed setups at http://www.assembla.com/spaces/scala-ide/wiki/SetupIt is a good idea to allocate Eclipse at least 1G – I use 2G.

8.If you changed any settings, restart Eclipse

To test your eclipse installation, select File/New/Project and see if “Scala Project” is in the list. You’ll use it for the “Hello World”exercise.

Check that the sources for the scala library are setup properly:

1.Open your project

2.CTRL+SHIFT+T to find types and type ‘List’

3.Scroll down the list to find the List from scala.collection.immutable

4.Click enter – you should now see the nicely formatted source code for List

5.You can take a quick look but don’t despair if that code looks foreign – we’ll get there soon enough.

Spruce up your Eclipse – my favorite plugins include:

-The GOTO plugin from, http://muermann.org/gotofile/

-The Color themes plugin, from http://www.eclipsecolorthemes.org/

-The VI plugin, from http://www.viplugin.com/viplugin/

Cheers – you are now setup to play with scala.

Racer kidz and wiki clubs - update

noreply@blogger.com (Razvan) — Fri, 24 Aug 2012 17:19:00 +0000

My little R&D into wiki domains at http://www.racerkidz.com seems to work fine - I add a feature evry now and then.

You can register a club, describe a racing schedule, then racers can add it to their profile and automatically, via the magic of wiki xpath, see the racing calendar and the map...

You can give it a try without an account from the home page.

I also experimentally used it to create simple blogs, like this enduro school. A blog has posts and that's that, see its domain definition.

Cheers!

Killing an annoying warning

noreply@blogger.com (Razie) — Mon, 29 Mar 2010 00:23:00 +0000

The most annoying warning ever has been haunting me for a while.


    [javac] /home/razvanc/workspace7u6/razxml/src/razie/base/data/RiXmlUtils.java:160: warning: com.sun.org.apache.xpath.internal.objects.XObject is Sun proprietary API and may be removed in a future release

I refuse to include and move around large unnecessary libraries to deal with XML and XPATH just because Sun considers themselves the center of the universe and their internal libraries somehow more important then mine. In short, they figured there should be no way to remove this warning, although, in this case, they copied code from apache into their own libraries.

Well, the solution is rather simple. Most likely everyone has specific XML wrappers. Just put them in their own project, disable the java/scala builders. Then, build it manually, create a jar file and check it in.

Then, in all the other projects, use this jar directly. Since the jar is already compiled, the annoying warning is no more.

See for instance my xml utilities, at http://github.com/razie/razxml

What's more, I don't actually use the XML stuff directly. I wrapped it all in pretty much one single class, using XPATH for access. It's simple, fast enough and makes for very simple code:


Reg.doc(MediaConfig.MEDIA_CONFIG).xpa(
  "/config/storage/host[@name='" + Agents.me().name + "']/media/@localdir"
)

For all xml data access, you should limit yourself to the xpe/xpl/xpa methods described in an earlier post (http://blog.homecloud.ca/2010/02/one-xpath-to-rule-them-all.html). So, use something like this instead: http://github.com/razie/razxml/blob/master/src/razie/base/data/XmlDoc.java

Cheers.

The Option monad pattern thing

noreply@blogger.com (Razie) — Sun, 14 Mar 2010 01:00:00 +0000

Having options is cool...even scala's Option[A]... :)

One way to use Option is plain:

addressMap.get("John") match {

case Some(addr) => popupMap (addr)

case None => logError(...)

}

or "java style" (synonym with "plain ugly"):

x = addressMap.get("John").getOrElse (null)

if (x != null) {

popupMap (x)

} else {

logError(...)

}

As you use scala more, you understand why monads are elephants and start doing it nice:

addressMap.foreach (popupMap(_))

addressMap.map (_.city)

And then you start to understand it as a pattern, basically you understand there's a range of problems were options apply. Say you create a small scripting framework (the same applies to say a command pattern framework). A script's execution may return successfully and with a value, or give a syntax error or a bunch of other options.

// the result of running a smart script

class RSResult {

def foreach (f:Any=>Unit) {}

def map (f:Any=>RSResult) : RSResult = RSUnsupported

def getOrElse (f: => Any) :Any = f

def jgetOrElse (f:Any) :Any = getOrElse(f)

}

case class RSSucc (res:Any) extends RSResult {

override def foreach (f:Any=>Unit) { f(res) }

override def map (f:Any=>RSResult): RSResult = f(res)

override def getOrElse (f: => Any) : Any = res

}

case class RSError (err:String) extends RSResult

object RSIncomplete extends RSResult // expression is incomplete...

object RSUnsupported extends RSResult // interactive mode unsupported

object RSSuccNoValue extends RSResult // successful, but no value returned

And then, of course, the methods returning these:

I used Any instead of [A] for simplicity, but there you have it. The Option monad pattern thing. Pretty cool!

Scripster - interactive scala REPL using telnet, http etc

noreply@blogger.com (Razie) — Tue, 02 Mar 2010 01:59:00 +0000

As mentioned before, I think that all apps must alllow scripted/programatic access to their objects.

I created a simple interactive gate to acces the scala REPL in any scala process. It only uses one port and supports telnet, http and swing:

The graphics are based on my 20widgets project, with the addition of a ScriptPad widget.

Content assist is only available in the telnet version (sic!). I will work to find some nice syntax-colored controls for the web version. Switch to character mode ("mode character" on ubuntu or default on windows) and use TAB to see a list of options. Right now it's a demo only, I need to find the parser's APIs to get the real content assist options ;)

You can download the single scripster-dist.jar file from my razpub project download, at http://code.google.com/p/razpub/ and run the example with this command line (replace /host/bin/scala with your $SCALA_HOME) :

java -classpath ./scripster-dist.jar:/host/bin/scala/lib/scala-library.jar:/host/bin/scala/lib/scala-compiler.jar:/host/bin/scala/lib/scala-swing.jar razie.scripster.JScalapSwing

To use it in your process, put these jars in your classpath and use this:

razie.scripster.Scripster.create(4445)

where 4445 is the port you want to use, see the razie.scripster.MainScripster for an example. To enable the swing version, use the swing jars as well and see the razie.scripster.MainSwingScripster class for an example.

For further tweaking, look at the code yourself, at http://github.com/razie/scripster/tree/master/src/razie/scripster/

If you want to keep in touch with the evolution of scripster or my other related endeavours, subscribe to the RSS feed or twitter/razie.

Scripster's main page, kept up-to-date, is at http://wiki.homecloud.ca/scripster. There's also an online live demo at http://scripster.codewitter.com.

Enjoy!

One XPath to rule them all!

noreply@blogger.com (Razie) — Sat, 06 Feb 2010 17:32:00 +0000

ABSTRACT: there's a million ways to access (data, objects, tables etc) these days and probably new ones are created every milisecond. What if we can figure out one that can be used anywhere?

Think about it!

/Universe/Planets/Planet[@name=='earth']/@position

XPath was created as a natural way to address elements in an XML document. One specifies the path - starting from the root node - to the element you are addressing. Conditions help you select the right nodes from many (lists) and you can address nodes or their attributes.

Extending it to addressing trees is strait-forward, so we can apply the same paradigm to any tree-structure, like Java beans...see for instance this Apache library: http://commons.apache.org/jxpath/

Come to think about it, all direct acyclical graphs fit in the same category...of course, with small differences: selecting an edge from among many possible types and lack of a root...so, some extensions are in order:

/Person[@name='John']/{hasFriends}Person

What hapens here is that the node of type Person that meets the criteria was selected as the start node and then the graph walk begins. From the many relationships tying humans together, the ones of type "hasFriends" are followed and not "hasChildren".

Yeah, you got it - even cycles are handled this way simply because as the path is walked, it reaches an end...

And... yeah, you got it again: UML domain model of classes with associations...it can describe any model you may use internally or present to clients...so we can handle pretty much any domain model described in an UML class diagram.

There.

I found myself needing four basic operations:

xpe : T - /** find one element */
xpl : List[T] - /** find a list of elements */
xpa : String - /** find one attribute */
xpla : List[String] - /** find a list of attributes */

For the sake of being concise, I will represent this new "data access interface"as either of the versions below (with explicit type and start node or implicit):

val me = XP[Person] xpe ("/Person[@areGroovy=='yeah, baby!']") from john

val minime = xpe ("/Person[@areGroovy=='yeah, baby!']")

So, why would this be useful?

Frankly, I'm growing old(er) and tired of all the non-sensical APIs that pop-up all the time everywhere: My application is unique! My domain model is complex! Everything could be an Object, but so what? I'm smarter! I know better!

My answer is: nope, your application's domain model is a class diagram and your actual objects thus form a graph. I don't need you to inven new ways to interact with your objects, over a million protocols and a million forms, when we can unify it all.

It's a very simple pattern. All it requires is that users understand basic modelling and the application's domain. The same expressions would be used by everyone...if not, at least the same pattern. No more gazillion interfaces and gazillion models.

Think about communication. Can be done in 2 ways:

1) blah blah blah can blah blah when blah blah and then some more blah blah

2) UML

You give me your domain model and, since you implement this simple no non-sense path-based model access, that's all I need to know. What you should focus on is documenting what the actual objects in your model: what they mean and what I can do with them once I have them.

This also enables generic graphical software creation. All I need is to have your domain model and then I can use a generic path composition tool (pick a node from a tree if you want).

It interactively explores the domain model, picking elements on the way.

In my integration work, I would love it if I didn't have to waste time writing stupidifying code over a miriad protocols just to get to what I needed.

If you're wondering how exactly the data should be accessed, we'll cover that in one of the next posts on this subject, in the mean time, take JOSH to heart: JSON, OSGI, Scala, Http...thank you, Gray Lens Man!

Enough for now - will continue soon...if you're interested and impatient, you can play with my prototype at http://github.com/razie/razbase/blob/master/src/razie/XP.scala

Counting the lines of code in a project

noreply@blogger.com (Razie) — Tue, 15 Dec 2009 21:56:00 +0000

cd ~/src
find . -name "*.java" -exec wc -l {} \; | awk '{ SUM += $1} END { print SUM }'

this is cool...

One Person

noreply@blogger.com (Razie) — Tue, 03 Nov 2009 17:54:00 +0000

Abstract: You are one person, using multiple devices - why shouldn't your stuff follow YOU rather than the device? Since we don't like depending on a 3rd party, the "home" cloud is the answer?

--------- Issue

Most programs today are isolated islands of functionality. The Web 2.0 transforms that, allowing you to mesh together pieces of functionality and information. The same is not true on personal computing platforms i.e. your laptop.

The bookmarks you store on your laptop can't be accessed when you're on the desktop at home and vice-versa. Same for your notes and pretty much anything, unless you manually copy them etc.

Option 1: store information on the web (centralised location). The one benefit of this is the ability to use that information from anywhere and any device. The drawbacks include big-brother and relying on a 3rd party for availability (including backup etc). Of course, the need to be online...although smart solutions can use local caches.

The big brother issue is getting worse. As much as you trust ALL levels of government of ALL countries that may have a claim on you or your property AND all their agencies, there are more and more corporations that mine all kinds of data about your person, for different reasons. Some as benign as offering you a new credit card and some as bad as credit recovery, setting insurance premiums etc. Let's not forget identity theft.

Option 2: use a home server. Basically, almost everyone has by now a server at home, which is constantly connected to the big Cloud, whether you're downloading movies or host your own blog. Why not use that instead of a 3rd party, to serve other stuff.

There are many a solutions to host your own stuff and access/share from/with the world. There's lots of software to serve security camera feeds, TV programs from your home, the Windows Home Server etc...the trend is already in place.

Having resolved (somewhat) the big brother and 3rd party dependency problem, there's still the issue of needing to be online.

Option 3: fully distributed and synchronised personal cloud. Store information locally, on the device that creates it (like on the work desktop where you save a favourite) and synchronise all devices (peer-to-peer or properly distributed solution). The only drawback is lack of on-demand availability of the information from other devices, until the sync occurs. This can be solved however by connecting your own PC to the net...or combining with option 1 or rather 2.

------------- Vision

The vision is that of you and your information following you. In fact it's not following you...it's just "there" for you to use. Whatever computer in your "home" cloud you use, your information is there.
I mean, will you upload ALL your favourites on facebook, just so you have them available in the living room, when you get home? I don't think so!

The solution is the logical conclusion of the same debate of distributed vs. centralised that's been raging for the past few decades. All those "in the know" know that it's a wave function. Now we may be heading towards the climax of centralised internet based clouds, but, as more PERSONAL processing power becomes more online, split between more types of personal devices and connected, the trend in the other way is just a matter of time...wave, right?

So, I dream of many agents, running on many personal devices, connected in your "personal" or "home" cloud, sharing all kinds of information and cooperating.

So, when you mark a "favourite" on your work desktop, it automatically gets replicated on your laptop, your home desktop (the cloud's gateway) and all the computers in the house, including the one in the living-room. There's nothing left to do but, when you get home, sit down and enjoy it on the big screen!

No 3rd party knows you enjoy that hardcore woodworking show or that you're trying to fix the toilet seat, no insurance company can mine that you once watched 3 illegal races on youtube etc...all usually available on your facebook or whathaveyou.

-------------- The social aspect

Today, no application is complete without considering the social aspect. You could obviously push some of your favourites to facebook, or share with friends in "friend" clouds.

-------------- Flexibility, customisation

Probably the biggest gain from a "central" cloud, like gmail etc is flexibility and customisability. Gmail is gmail is gmail. That's exactly what it does and, while you may change it's fonts and even access it remotely via an IMAP or http API (thank you, Google) that's still exactly what it does.

A personal cloud, though, can be customised. You decide what it does and how it does it.

-------------- try it

If you want to play with such an agent, try a preview of mine, at http://razpub.googlecode.com/downloads

Read more about it at http://wiki.homecloud.ca

You can even try the remote favourites sharing prototype...follow this link after you downloaded, installed it and started it on several computers: http://localhost:4444/mutant/capture.html - After capturing, go to another computer in your home cloud and see the links at: http://localhost:4444/mutant/asset/Link

This is written as of version 0.x so the links may change - the use case however will be maintained up-to-date in the wiki, at http://wiki.homecloud.ca/savedlink

Modern Applications

noreply@blogger.com (Razie) — Fri, 02 Oct 2009 12:53:00 +0000

ABSTRACT: today it is unacceptable to have any application built as an independent islands of functionality. It is necessary to expose functionality via simple protocols (http).

Really, it is unacceptable today to have any application built as an independent piece of functionality. No application is or should be complex enough to do EVERYTHING. Thus, since we do not view our users as the mandatory human servers of the omnipotent computer -- with the noble lifetime goal of clicking through the menus, dialogs, wizards and buttons that the all-mighty programmer has imposed on them -- all application components NEED TO OFFER API-based access to a basic set of services/functions/objects/whathaveyou.

These span a large number of functional areas, from controlling the application remotely, scripting its behavior, input/output/logging etc, so we'll take them one-by-one in future posts.

Point being that applications become interoperable and subject to automation. There's no other way really to automatically turn down the volume of the DVD player because you answered a call on skype! OR have a tweet on your cellphone when the torrent has finished donwloading the latest CBC show.

--- Interoperability == standardisation?

This thinking is normal to any Unix hacker, since they allways did "find|grep|cut|wc|sort" but it seems too complicated for anybody else to comprehend, especially window-heads.

With physical devices, standardisation is, granted, not easy. Standards arise from a need which is already fulfilled by existing devices, cables and plugs. Then the plugs get standardised and devices become interoperable.

The same was true in the software world (find|grep), but it's been false for a long time, especially since the advent of the stupidifying mouse and the un-parseable pixel.

Command lines have always been around and http has also been around for a long time. There really is no reason not to combine both, when all you want is users to use the functionality your code has to offer.

Ideally, all applications would be certified as "open" and "interoperable" and we will get there, in time. DLNA/UPNP is an example of such an interoperable framework, including certification. So is OSS/J (a set of telecom APIs) and others.

--- using http at rest to describe and interact with the object-oriented world

So, what should ideally happen? We'd have unified protocols for access and interoperability. HTTP, telnet, command lines come to mind. Formats are also an obvious need: HTML/XML/JSON/text. Let REST dictate how things work, but only a modicum of human intervention can be mandated.

Beyond the needs, there's the wants. We'd like that each would expose their internal models (objects, services, methods, actions, functionality) in a standard way, parseable and understandable by others.

If you extend that to the interoperable web, you get the semantic web. Well, almost, since that's been designed by DBAs - they'd like to call it the "data web" and hide logic in a view that offers more data :). Sorry, couldn't help it!

I live in a world surrounded by objects I'm interacting with. Data/information has a very important role to play, but it's not the end-all. My newspaper can not only filter for me the latest developments on that accident, but can also manage my account and micro-payments. Having knowledge of an intermediary PayPal only serves to confuse me and keep my mind busy with concepts I don't care about, since i already told my "gate keeper" that I trust my newspaper somewhat...

The semantic web guys have it right, though. Exposing standardized schemas of the applications's objects and functionalities is the way to go. Scoping these (my "train" is different than your "train") makes obvious sense (think namespaces).

---- So, vat do you vant?

In the B2B universe (now turned SOA) there's lots of WS-based standards, including security, identity exchange etc. These don't bode well on the web, though. Why write things twice? It has to be simple-over-http. PERIOD, dumby.

So, each modern application has:
- embedded http server
- exposing functionality/internal models etc
- designed to be interoperable

I would like everyone to stop buying and using any application that doesn't meet these criteria. Harsh, but then i'm in a bad mood today...

------ So, what's a developer to do?

Embed an http server. In case you need a small, lightweight embedded web server, checkout my public project at http://razpub.googlecode.com . There's lots of others out there - actually I recommend you get one that supports the servlet standard and you do servlets.

Think about and define your business model, spell out your domain entities. Use less services and more objects when defining your API and bode nicely with REST.

Define your model in whatever format you want - just make sure it's an xml file :). We'll deal with these in a future post, but for now it must be objcet-oriented: each 'class' has 'attributes' and 'methods'. The methods have a name and a list of arguments. Keep all types to String for now...assume all interaction is via URLs, which can't marshal bytecode - that's something we'll have to deal with.

Offer access to the entire business model via http, including content (values) and control (invoked methods).

Document all this very nicely in an embedded set of html pages. No smarts neccessary. Simple solutions always work better than complex ones.

If you're looking for an asset/modelling/http object interface framework, checkout the com.razie.pub.assets package of my razpub project.

Stick to the REST principles for now, until my "REST is bad" post is posted.

Modularity - allow extensions of functionality. Since the future is OSGi, may I suggest an OSGi compliant server - mine will be but it's not.

And generally, don't forget to have fun!

P.S. Just to give a concrete example, the VLC player is a great one. It has several interfaces, including telnet and http and you start whichever you want...

What's up with the home cloud?

noreply@blogger.com (Razie) — Thu, 04 Jun 2009 15:58:00 +0000

ABSTRACT: An abstract description of what I'm up to here.

I am generally frustrated with the state of software - and have been, for a long time. People love to reinvent the wheel all the time, make life miserable for the users that don't stick to base use cases etc.

I have some suggestions on how applications should be written and how the (software) world should look like. This is of course just one of the directions in which we could evolve but it's the one that seems to me to make the most sense.

There are quite a few components to this picture so you'll have to bear with me over the course of several separate blogs on related topics and, at the end, we'll put everything together.

There will be three different levels of this new "architecture", bottoms up:

1. The basic idea is that there are certain things that are necessary to include in ANY software component of any kind, for instance exposing an API to access and manage the internal objects, data and functionality.

2. A generic distributed application support framework will then use these to generate the "home cloud" - a cloud of interacting devices and objects under your personal, direct and total control.

3. From here, moving to the web, you can have clusters of clouds of different functionalities, completely removing the necessity for centralized software.

4. On top of these, there are different functionalities and specific use cases, such as "using identities", synchronizing files etc. I am not happy with the way these are handled by existing software, so you'll get a chance to hear my opinion.

It will take me a while to put each into words and sample code. I'm not just blogging here, I'm also putting together a framework to test all these concepts, hopefully good enough that it will be used by others. You can get a glimpse and keep tabs on that here http://razpub.googlecode.com

I will try to blog at least once a month, so a somewhat complete picture will form this year, but don't hold either your breath or me to it.

Intended audience: mixed. The blogs are individually searchable, so a personal review of the "Windows Live Sync" for instance will benefit anyone searching for that kind of stuff. I will try to categorize each blog, to benefit the followers.

I tend to be terse, so be ready to fire up your brain and read between the words. Non software-development professionals may have a hard time with my terseness and techie wor(d) games...they'll figure it out fast and stop reading.

I think that's all for now - I'll end this here and start working on the next month's blog.

Have fun!