Scriptedbrowser

A tool to automate a browser and create a movie recording of it.

Installation

as external dependies the following tools are needed:

• a VNC server, tested with TigerVNC, others should work
• xdotool a tool to create mouse/keyboard event
• firefox with geckodriver as the browser to control
• ffmpeg to record the video

for developing, we recommend that you install phan by using the included composer file. composer install

Usage

If you start a new session, just call it via php index.php script_to_run.json It will print out a session, that you should use for the next runs php index.php script_ro_tun.json session_id

Script file

A script file is a JSON file. It contains one or more plans to execute. Each plan has some base parameters and list of actions to do:

{
  "plans": [{
    "id": "register",
    "video": "register.mp4",
    "mousespeed": 500,
    "typespeed": 5,
    "framerate": 30,
    "vars": {
      "baseurl": "https://contentnation.net",
      "urlprefix": "${baseurl}/${lang}",
      "lang": "en"
    },
    "actions": [
    ]
  }]
}

Each plan has an id, which is an internal name, an output video file, a video frame rate and default speeds for mouse and typing. Additionally you can define variables that can be used later in the actions block. An example action block is given here:

"actions": [
  "url ${urlprefix}/",
  "sleep 2",
  "startrecording",
  {"name": "mousemove", "parameter": "#reglogin", "duration": 3},
  "sleep 1",
  "leftclick",
  "waitforurl ${urlprefix}/login.htm",
  "mousemoverel 0 1",
  "mousemove a[href=\"/${lang}/register.htm\"]",
  "sleep 1",
  "leftclick",
  "waitforurl ${urlprefix}/register.htm"
]

Each command can be described in 2 ways. As a string with name and parameter or as a json when more parameters should be given.

Valid commands

• injectscript URL injects a javascript into the current browser and page
• leftclick triggers a left mouse button click at current position
• mousemove cssselector move mouse to element with given css selector, default speed is given in block parameters, but can be overwritten by local mousespeed parameter or by given a duration in seconds to reach the goal.
• mousemoverel x y moves mouse relative to current position by x y pixels
• sleep time will sleep for given amount of seconds.
• startrecording start the recording using the given frame rate and output file in the main block.
• stoprecording finishes the recording.
• type text types in the given text
• url URL changes browser URL to given new value. This will load the page.
• waitforurl URL waits until given url is loaded, useful after a click on a link or form

Development and extending

Additional commands

You can extend the tool with own commands. Each command needs to be an own file in the scriptedbrowser/commands directory. One such file is included, named dbquery.php and the matching command dbquery.

A minimal example:

<?php
namespace ScriptedBrowser\Commands;

/**
 * inject and execute given javascript
 *
 * @param \ScriptedBrowser\Main $main main instance
 * @param \ScriptedBrowser\Control $control control interface
 * @param array<string,mixed>  $options options
 * @return bool true on success
 */
function dbquery($main, $control, $options)
{
    return true;
}

In the function you can do whatever your scripting needs desire. You can also set variables to be used later by calling $main->setVar("name", "value") and using ${name} in the script later. Options from the script are given via $options.

Troubleshooting

can't open VNC session.

Make sure you can start vnc sessions and connect to it via the normal tools. Sometimes you need to define passwords or the server does not start.

can't record video, resolution is wrong

This depends on the vnc server, but at least for TigerVNC, create a file config in ~/.config/tigervnc containing the line geometry=1920x1080