Monday, January 14, 2013

Stateful, but virtually stateless, thanks to event sourcing




My example application of MFlow  has two handles for two verbs: mainmenu and shopCart

addMessageFlows [(""  ,transient $ runFlow mainmenu)
                ,("shop"    ,runFlow shopCart)]



mainmenu has a set of links to different options, but because the flow is stateful, theoretically I can not address each individual option with a link:


mainmenu=   do
       setHeader stdheader
       setTimeouts 100 0
       r <- ask $ wcached "menu" 0 $ 
               wlink CountI   (b << text "increase an Int")
               <|> (p <<< wlink CountS   (b << text "increase a String"))
               <|>  br ++> wlink Action   (b << text "Example of action, executed when a widget is validated")
               <|>  br ++> wlink Select   (b << text "select options")
               <|>  br ++> wlink CheckBoxes (b << text "checkboxes")
               <|>  br ++> wlink Radio    (b << text "Radio buttons")
               <++  br <>  br <> b << text "DYNAMIC WIDGETS"
               <|>  br ++> wlink Ajax     (b << text "AJAX example")
               <|>  br ++> wlink Autocomp (b << text "autocomplete")
               <|>  br ++> wlink AutocompList (b << text "autocomplete List")
               <|>  br ++> wlink ListEdit (b << text "list edition")
               <|>  br ++> wlink Grid (b << text "grid")
               <|>  br ++> wlink TextEdit (b << text "Content Management")
               <++  br <>  br <> b << text "STATEFUL PERSISTENT FLOW"
                 <> br <>  a ! href "shop" << text "shopping"   
                 <> br <>  br <> b << text "OTHERS"

               <|>  br ++> wlink Login    (b << text "login/logout")


       case r of
             CountI    ->  clickn  0
             CountS    ->  clicks "1"
             Action    ->  actions 1
             Ajax      ->  ajaxsample
             Select    ->  options
             CheckBoxes -> checkBoxes
             TextEdit  ->  textEdit
             Grid      ->  grid
             Autocomp  ->  autocomplete1
             AutocompList -> autocompList
             ListEdit  ->  wlistEd
             Radio     ->  radio
             Login     ->  login
I have to go to the menu first to reach any option, because mainmenu ask for the menu in the first place. Ever. But this is not that way. Each option has an absolute URL and can be accessed directly from outside. It can be copied and mailed safely to other user. This link bypass the menu and go straight to the first option: http://81.169.134.95:8081/noscript?c0=CountI It may be said that, because MFlow simulates the structure of a console application, to reach the second question it is necessary to answer  the first question. But MFlow allows to express the sequence of events (the user responses) of various steps in a single URL. In the above URL,
 c0=CountI 
is the response to the menu question, so the first 'ask' will read the menu response right from the link url. It is possible to express by hand as many user responses in a single URL as I would like, but the application now only generates links with the events corresponding to the last interaction  if the request was in GET mode. That is enough to generate a link that hop over a first step of links straight to the desired option. This example has events for two steps. it chooses the "Content Management" option and  execute the first step of this option that present a login form. But this URL has been generated by hand: http://81.169.134.95:8081/noscript?p0=()&c11=TextEdit But it is may be easy to develop an 'addressable' switch that, when activates, generates addressable URLs. It also would reset parameter numbering, so that even POST requests  would navigate to pages with a addressable URL and the page will be addressable and shareable as if the flow where stateless. With this event sourcing approack,  an stateful MFlow procedure could be as stateless (or even more) than a classical web procedure.

NOTE: by design, a link to mainmenu with no parameters will go to the current user navigation point, rather that going back to the root of the navigation (the menu). 

If there is a timeout, set with setTimeout, like in this case of 100 seconds, after this time the link to mainmenu with no parameter will go to the root.

Online MFlow demo

There is an online site where MFlow is running with some examples of some MFlow features:

http://mflowdemo.herokuapp.com

This is the frontpage:


MFlow examples


About this menu (article)

DATABASE
  • Database Create, Store and retrieve lines of text from Amazon SimpleDB storage

  • PUSH
  • Push example A push widget in append mode receives input froma text box with autorefresh (article)
  • A push counter Show a countdown. Then goes to the main menu (article)


  • ERROR TRACES
  • Execution traces for errorsproduces an error and show the complete execution trace (article)


  • DIFFERENT KINDS OF FLOWS
  • REST navigation Navigates trough menus and a sucession of GET pages (article)
  • Stateful flow: shopping Add articles to a persistent shopping cart stored in the session log.getSessionData is read in the View monad to get the most recent shopping carteven when the back button has been pressed (article)
  • Stateful flow: Counter a persistent counter. It uses the same mechanism than shopping, but it is a more simple example


  • BASIC
  • Increase an Int A loop that increases the Int value of a text box
  • Increase a String A loop that concatenate text in a text box
  • Select options A combo box
  • Checkboxes
  • Radio buttons


  • PAGE FLOWS with MONADIC WIDGETS, ACTIONS & CALLBACKS
  • Example of action, executed when a widget is validated
  • in page flow: sum of three numbers Page flows are monadic widgets that modifies themselves in the page (article)
  • Counter A page flow which increases a counter by using a callback (article)
  • Multicounter Page flow with many independent counters with autoRefresh, so they modify themselves in-place (article)
  • Combination of three dynamic widgets Combination of autoRefreshe'd widgets in the same page, with different behaviours (article)
  • Modal dialog A modal Dialog box with a form within a page flow


  • DYNAMIC WIDGETS
  • AJAX example A onclick event in a text box invokes a server procedure that increment the integer value (article)
  • autocomplete Example of autocomplete, a widget which takes the suggested values from a server procedure
  • autocomplete List Example of a widget that generates a set of return values, suggested by a autocomplete input box (article)
  • list edition Example of a widget that edit, update and delete a list of user-defined widgets
  • grid Example of the same widget In this case, containing a row of two fields, aranged in a table (article)
  • Content Management Example of the (basic) content management primitives defined in MFlow.Forms.Widgets


  • OTHERS
  • login/logout Example of using the login and/or logout
  • Prevent going back after a transaction Control backtracking to avoid navigating back to undo something that can not be undone. For example, a payment (article)
  • Thursday, January 03, 2013

    MFlow active widgets example


    The new module MFlow.Forms.Widgets in the package MFlow  whose last version is in GtHub, (reference below) contains active widgets that can manage other widgets in order to compose complex dynamic applications. A widget is composed of form elements, links and scripts that return statically typed results.

     Now, an example of use of the active widget wEditList, described here.  This widget can manage a list of widgets of the same type. It can add or delete them. The result, when the list is sent to the server, is the sublist of the validated elements results.

    In this case, the widget managed by wEditList is a row of a table with three elements: an Int input text, a String input text and a delete link. the widget is codified using blaze.html and the MFlow input fields.

    The delete link  invokes a javascript event that delete the entire row.

    There is a "add" link that add a new row to the list. To be recognized by the wEditList widget, it has to have the identifier "wEditListAdd"



    grid = do
      let row _= tr <<< ( (,) <$> tdborder <<< getInt (Just 0)
                             <*> tdborder <<< getString (Just "text")
                             <++ tdborder << delLink)

          delLink= a ! href "#"
                     ! onclick  "this.parentNode.parentNode\
                        \.parentNode.removeChild\
                        \(this.parentNode.parentNode)"
                     << text "delete"
          tdborder= td ! At.style  "border: solid 1px"

          addLink= a ! href "#"
                     ! At.id "wEditListAdd"
                     << text "add"

    Note the use of (,) and the applicative operators <$> and <*> over the form elements getInt and getString, which
    compose a result of type (Int,String). There are also formatting operators <<< which encloses the form elements within a table colum element 'tborder' which is a 'td' element with some style. The ability to mix formatting and applicative operators is a unique feature of MFlow. It becomes quite intuitive to use after a while. A knowledge of  applicative operators is necessary. While <<< encloses a form element (or a wlink) in a html tag,  << encloses ordinary html, and this is why dellink, an ordinary html link is preceded by <<  . Finally the operator <++ append html formatting to a widget.

    Here is how the widget looks with two initial elements:


    add
    delete
    delete

    Submit

    This is how the wEditList is invoked with the row element as parameter. the 'table' holds the elements in this case. A initial list of two elements with empty strings,that are ignored, means that two elements will be show at the beginning. Finally, a submitButton will send the form result when pressed. To glue these components, the operator ++> prepends html to the widget and the operator <** is an applicative operator similar to <* which do not care about the validation result of the button (<* could have been used in this case). See Control.Applicative for the Applicative concept and the applicative operators.

    r <- ask $ addLink ++> ( wEditList table  row ["",""]) <** submitButton "submit"

    r will have the result. When I click add, two times and  edit some of the fields, the look is as follows:

    add
    delete
    delete
    delete
    delete


    Submit

    if we add an additional line to print the result in a new web page. after pressing the submit button:

    ask $   p << (show r ++ " returned")


    The page will show this result:

    [(1,"text1"),(2,"text2"),(3,"text3"),(4,"text4")] returned

    The complete code is part of demos.Blaze.hs. 


    This example uses the last version of MFlow at https://github.com/agocorona/MFlow.
    It uses the last version of  Workflow  https://github.com/agocorona/Workflow


    grid = do
      let row _= tr <<< ( (,) <$> tdborder <<< getInt (Just 0)
                              <*> tdborder <<< getString (Just "text")
                              <++ tdborder << delLink)
          addLink= a ! href "#"
                     ! At.id "wEditListAdd"
                     << text "add"
          delLink= a ! href "#"
                     ! onclick
                        "this.parentNode.parentNode\
                        \.parentNode.removeChild\
                        \(this.parentNode.parentNode)"
                     << text "delete"
          tdborder= td ! At.style  "border: solid 1px"

      r <- ask $ addLink ++> ( wEditList table  row ["",""]) <** submitButton "submit"
      ask $   p << (show r ++ " returned")
          ++> wlink () (p << text " back to menu")


    Sunday, December 30, 2012

    On the "spirit" of MFlow. Anatomy of a Widget


    TCache, RefSerialize, Workflow and MFlow are packages created in the process of building a system for massive workflows in the Web. It could be used for electronic democracy among other applications. MFlow  has been designed to program Web applications at a high level that I need by sacrificyng stability, completeness and some simplicity. It is also based of the idea of self contained code, composability and to encapsulate as much "magic" as I can.

    Haskell can do a lot of magic in MFlow: It can express a web navigation flow in a way that in Java would need a configuration file. it can run a web server procedure backward to respond to a previous page in case the browser back button was pressed.  It can free resources by killing a process on timeout, because it can restart the server process when necessary and recover the computation state. It can get rid of database storage  and data storage definitions because all is re-created from simple events. It can compose statically typed pages including links and scripts. All these are unique characteristics that are unique in MFlow, but also it is possible a classical stateless application, with data storage defined by the user in a database.

    Since the workflow concept need a way to checkpoint events,  The Workflow package has a monad that store the internal and external events, and restore its state from them without the need for an additional snapshoot of the  current state, which  indeed would need an additional data definition beside the definition of the events. That is what is called "event sourcing".

    MFlow uses the hability of Workflow to store and rebuild the state of the data from the events, in this case, produced by user interactions in the Web. This means that there has no need for a data layer if you don´t want it. Just the automatically generated log of events is enough. This advantage becomes a necessity when a data definition of the current state is not only redundant but impossible without an additional interpreted DSL. For example I want to use MFlow for a functionality in which the user define processes by means of a wizard-like interaction, with some questions asked to the user. The generated description is a program that has no direct data serialization except by means of a interpreted DSL that would be painfully complicated, and would not add any additional functionality except the intended persistence of the current state. Instead,  this programming overhead can be avoided by using event sourcing. See here.

    RefSerialize is a serialization library that reduces the size of the event log by referencing multiple repeated events to a single description in the log file. TCache gives transactionality and read/write caching, so that log writing and reading is done in bursts according with the cache write policy, and in coherence with other kind of non-event data that may be used by the application.

    As I said before, the requisite of self contained-ness for the creation of components is very important for me. It means that the habitual mess of configuration files, heterogeneous dscriptions, templatings, scripts, all of them in different files that make necessary a deployment step and a complete manual for reusing it is avoided. You can still have your separate  header-footer thenplate in a different file, codified using XML,  but this is not imposed. You can have all of this in the same file.

    Here below is an example of a reusable widget in MFlow. A widget is an active component. It ever return something statically typed, verified at compilation time. The output may be the result of a click in a link, a form submit or a script. It can have Ajax interactions.  It can use/contain other widgets or combinations of widgets that interact with the server server trough ajax.It uses Ajax to call 'prependWidget', that insert a new empty widget of type 'a' on top of the list. The widget 'requires' the installation of  a short 'ajaxScript' that is inline, and the jquery file that is referenced by the URL. When jquery is loaded, a online script,  'installevents' is called and install the ajax event.




    wEditList :: (Typeable a,Read a
                 ,FormInput view
                 ,Functor m,MonadIO m, Executable m)
              => (view ->view)
              -> (Maybe String -> View view Identity a)
              -> [String] -> View view m  [a]
    wEditList holderview w xs = do

        let ws=  map (w . Just) xs
            wn=  w Nothing
        id1<- genNewId
        let sel= "$('#"<>  B.pack id1 <> "')"
        callAjax <- ajax . const $ prependWidget sel wn
        let installevents= "$(document).ready(function(){\
                  \$('#wEditListAdd').click(function(){"++callAjax "''"++"});})"

        requires [JScript ajaxScript, JScriptFile jqueryScript [installevents] ]

        ws' <- getEdited sel

        r <-  (holderview  <<< (manyOf $ ws' ++ map changeMonad ws)) <! [("id",id1)]

        delEdited sel ws'

        return  r



    'prependWidget' stores the added widgets in the edited list, that is accessed with 'getEdited'.  This last is a shorhand for 'getSessionData'', which stores and retrieves user-defined data in the session, in this case the inserted widgets, by means of a map indexed by data type. In this way the user don´t need to pass its application data by parameters, neither it need to create its own state monad. 'getEdited' just uses its own type for storing widgets.

    Finally, the value returned by the widget  is 'manyOf' the widgets added "ws'" and the initial widgets, "ws".  'manyOf' return only the validated widgets of the list. 'holderview' is the tag that encloses the list of edited widgets. the operator  '<<<'  add an encloser tag to the widget, and the operator '<!' gives atributes to the topmost tag of the widget, so the tag 'holderview' get assigned the  tag identifier 'id1'.  id1 is a name automatically generated. This is necessary for 'prependWidget' to add a new widget to 'id1' because his first parameter, the selector 'sel'  points to 'id1'.

    Because the edited widgets are internally cached with 'wcached'  they must be in the Identity monad. to switch to the 'm' monad, 'changeMonad' is used. In the future, probably I will use IO only and disallow the use an arbitrary monad, since the user can store in session arbitrary data thanks to 'setSessionData' so there is no need for an arbitrary monad. That would make the types in MFlow less intimidating for beginners, and reflect the effort on simplicity without sacrificing power.

    Finally the Edited list is deleted.before returning the result.

    This widget is used in "demos/demos-blaze.hs" and is part of the MFlow release. I´m programming reusable active widgets general enough to be used in any kind of application. 'prependWidget' is an example of server-side control within the widget, to add dynamic behaviour. There is a 'ajaxSend' primitive that permits to send/receive many ajax messages during the same ajax procedure. It is intended that with this kind of server-side control, a single page with widgets coordinated in this way can present complex behaviours to the user.

    The result is that the programmer can define general widgets at a level equal or higher than  ASP.NET server side controls or the JavaServer Faces with type safety. The price of the deep-first approach for accumulating levels of programming as fast as possible is the lack of stability (in fact the code above has changed) but the benefit is that the design updates will not be based on artificial tests, as a result of some assumptions, but in the use of  real application with real users.
    In the next post I will show some usage examples of this widget 
    This example uses the last version of MFlow at https://github.com/agocorona/MFlow.
    It uses the last version of  Workflow  https://github.com/agocorona/Workflow
    
    

    Wednesday, November 14, 2012

    MFlow now supports blaze-html

    MFlow now supports blaze.html.

    This example  uses Blaze.Html . It is almost identical to the Text.XHtml version.

    It show almost all of the functionalities of MFlow, so it has a lot of imports. Here the header and the main function is displayed:

    {-# LANGUAGE OverloadedStrings, ScopedTypeVariables, DeriveDataTypeable, NoMonomorphismRestriction #-}
    module Main where
    import MFlow.Hack  -- hiding (ask)
    import Hack.Handler.SimpleServer
    import MFlow.Forms.Blaze.Html
    import MFlow.Forms.Widgets
    import MFlow.Forms.Ajax
    import Text.Blaze.Html5 as El
    import Text.Blaze.Html5.Attributes as At hiding (step)
    import Text.Blaze.Internal(text)
    import Data.String
    --import MFlow.Forms.Test
    import MFlow
    import MFlow.FileServer
    import MFlow.Forms.Ajax
    import MFlow.Forms.Admin
    import MFlow.Forms
    import Data.TCache
    import Control.Monad.Trans
    import Data.Typeable

    import Control.Concurrent
    import Control.Exception as E
    import qualified Data.ByteString.Char8 as SB
    import qualified Data.Vector as V
    import Data.Maybe
    import Data.Monoid

    --test= runTest [(15,"shop")]

    main= do
       syncWrite SyncManual
       setFilesPath ""
       addFileServerWF
       addMessageFlows [(""  ,transient $ runFlow mainf),
                        ("shop"    ,runFlow shopCart)]
       run 80 hackMessageFlow

       adminLoop

    stdheader c= html << body << (p << text "You can press the back button" <> c)

    data Options= CountI | CountS | TextEdit |Shop | Action | Ajax | Select deriving (Bounded, Enum,Read, Show,Typeable)

    mainf=   do
           setHeader stdheader
           r <- ask="ask" br="br" nbsp="nbsp"> wlink TextEdit (b << text "Content Management")
                   <|>  br ++> wlink Shop (b << text "example of transfer to another flow (shopping)")
                   <|>  br ++> wlink CountI (b << text "increase an Int")
                   <|>  br ++> wlink CountS (b << text "increase a String")
                   <|>  br ++> wlink Action (b << text "Example of a string widget with an action")
                   <|>  br ++> wlink Ajax (b << text "Simple AJAX example")
                   <|>  br ++> wlink Select (b << text "select options")
                   <++ (br <> linkShop) -- this is an ordinary XHtml link


           case r of
                 CountI    ->  clickn 0
                 CountS    ->  clicks "1"
                 Action    ->  actions 1
                 Ajax      ->  ajaxsample
                 Select    ->  options
                 TextEdit  ->  textEdit
                 Shop      ->  transfer "shop"
           mainf

           where
           linkShop= a ! href  "shop" << text "shopping"




    This example uses the last version of MFlow at https://github.com/agocorona/MFlow.
    It uses the last version of  Workflow  https://github.com/agocorona/Workflow

    MFlow: now the widgets can express requirements

    A widget can need the installation of a client script, a CSS or download them. Also it can need a server process installed. But other widgets in the same page could need the same script. To avoid duplications, and to make easy the development of separated widgets maintaining the modularity, I added requirements.

    type Script= String
    type OnLoadScript= String
    type File= String
    data WebRequirement= CSSFile String
                       | CSS Script
                       | JScriptFile File [OnLoadScript]
                       | JScript Script
                       | ServerProc (String, Token -> Workflow IO ())

    [OnLoadScript] are scripts called when the script file is loaded.

    The syntax is as such:


    ask $  requires[WebRequirement] 
        >> rest of the page

    here >> is the monadic operator



    For example this widget show set of options in from data from the serves via AJAX,  is defined in MFlow.Forms.Widgets, insert two javascript files, a CSS file and  a script. It can be combined with other widgets that require also these scripts, but the page creation process will just insert a single script tag or css link. The programmer no longer has to care about the requirements of each widget


    selectAutocomplete serverproc = do 
        requires [JScript ajaxScript 
                 ,JScriptFile jqueryScript [events]
                 ,CSSFile jqueryCSS
                 ,JScriptFile jqueryUi []]
                 
        ajaxc <- ajaxcommand="ajaxcommand" attr="attr" font="font" text1="text1" value="value">
                             $ \u -> do
                                     r <- font="font" serverproc="serverproc" u="u">
                                     return $ jaddtoautocomp r

        getCheckBoxes
          (thediv ! [strAttr "id" "users"] <<< noWidget )
          <++ input ![thetype "text"
                     ,value "select users"
                     ,strAttr "id" "text1"
                     ,strAttr "oninput" ajaxc
                     ,strAttr "autocomplete" "off"]



    This example uses the last version of MFlow at https://github.com/agocorona/MFlow.
    It uses the last version of  Workflow  https://github.com/agocorona/Workflow

    Monday, November 12, 2012

    Content management in MFlow


    I Just added some templating/content management to MFlow that IMHO is more flexible and simple than proper templating or content management approaches.

    tFieldEd key html

    is a widget that display the content of  html as is, But if logged as administrator, it permits to edit this chunk of html in place. So the editor could see the real appearance of what he write in the page while editing. When the administrator double click in the paragraph, the content is saved and identified by the key. Then, from now on the users will see the content of the saved paragraph, not the original one in the code.

    The content is saved in a file by default ("texts" in this versions), but there is a configurable version (tFieldGen). The html content and his formating is cached in memory, so the display is very fast.

    In case that the content has been fiixed and it don´t need further editions, 

     tField key

    just read and present the edited content. tFieldEd is not longer needed.


    There are also multilingual content management primitives mField and mFieldEd 


    This demo shows in four steps how a typical process of content edition would work.

    textEdit= do
        setHeader $ \html -> thehtml << body << html


        let first=  p << italics << 

                       (thespan << "this is a page with"
                       +++ bold << " two " +++ thespan << "paragraphs") 
                       
            second= p << italics << "This is the original text. This is the second paragraph"
             
            pageEditable =  (tFieldEd "first"  first)
                        **> (tFieldEd "second" second)
            
        ask $   first
            ++> second
            ++> wlink () (p << "click here to edit it")
        
        setAdminUser "admin" "admin"
        
        ask $ p << "Please login with admin/admin to edit it"
                ++> userWidget (Just "admin") userLogin
        
        ask $   p << "now you can click the field and edit them"
            ++> p << bold << "to save the edited field, double click on it"
            ++> pageEditable
            **> wlink () (p << "click here to see it as a normal user")
            
        logout
        
        ask $   p << "the user sees the edited content. He can not edit it"
            ++> pageEditable   
            **> wlink () (p << "click to continue")
          
        ask $   p << "When text are fixed,the edit facility and the original texts can be removed. The content is indexed by the field key"
            ++> tField "first" 
            **> tField "second" 
            **> p << "End of edit field demo" ++> wlink () (p << "click here to go to menu")


            


    This example uses the last version of MFlow at https://github.com/agocorona/MFlow.
    It uses the last version of  Workflow  https://github.com/agocorona/Workflow