AI2 algorithm Arduino software

App Inventor (AI2) Joystick User Control

App Inventor (AI2) is an easy way to create Android apps. While originally designed for teaching, it still allows building advanced and complex apps, and was the tool I chose to build a software 2-axis joystick interface for wireless vehicle control.

This article describes the implementation of the joystick as a standalone component that can be integrated into other applications.

The demo application is for an on-screen 2-axis joystick. The displayed representation is a basic view of a joystick – the thumbstick is represented by a solid colored dot and the limits of the joystick are the grey colored area.

The demo also shows different options customising joystick self-centering on release to suit the specific application.


All the code discussed in this article can be obtained here as a loadable AI2 import file.

Coordinate Systems

We naturally think of a joystick moving in the X (horizontal) and Y (vertical) directions. However, the coordinates for an image on the AI2 canvas control don’t correspond to these directions. Canvas coordinates have their origin in the top left corner and increase horizontally to the right for X and vertically down for Y. This is shown in the figure below..

As our virtual joystick is centered around an arbitrary point in the canvas control, to get useful joystick coordinates (x, y) we need to shift the frame of reference to the centre of the joystick.

This is done by subtracting the center coordinates from the position on the canvas:

  • x = (CanvasX – JoystickCenterX)
  • y = (CanvasY – JoystickCenterY)

You will see this occurring in several places in the demo code, described below.

Tracking (Global) Variables

A number of global variables are used to track important information about the virtual joystick.

  • jsOuterRadius is the radius of the joystick. This sets the outer limit for the movement of the thumbstick circle and is calculated from the size of the canvas.
  • jsCenterX, jsCenterY are the canvas coordinates of the center of the joystick.
  • jPosX, jPosY are the current joystick thumbstick position, in the joystick coordinate frame.

Related to the joystick but not strictly required for the joystick control are

  • jsPctFullRange is the fraction of jsOuterRadius for the current thumbstick position, as a percentage.
  • jsAngle if the current angle of the thumbstick relative to the top of the joystick (vertical axis).
  • jsPctFullRangeOld, jsAngleOld are used to track when changes to the related current (not-old) coordinate variables occur.


Initialization of the joystick is done in the ResetJoystick procedure. This clears the canvas, initializes the global tracking variables and draws the joystick on the display with the joystick thumbstick in the home position.

SizeJoystickDisplay initializes the variables used to switch coordinate frames and manage the motion limits for the thumbstick. The center of the joystick is placed at the center of the AI2 canvas and the diameter of the joystick is the smallest dimension (X or Y) of the canvas size. This allows the size of the joystick to be centered and maximized in the available area.

DrawJoystick draws the joystick limit and home position as a sequence of concentric circles in the canvas. These are drawn from the outside in as the circles can only be drawn ‘filled’ and so need to overwrite each other to remain visible in the end product.

Finally, the whole process is kicked off when the AI2 screen is initialized. It is worth noting that I had some intermittent issues with the canvas sometimes not initializing properly, so I included a second delayed initialization by setting a timer to re-run the reset code after the initialize method had completed. This fixed my problem but may be unnecessary in your own applications.

At the end of this process the application will display a joystick control similar to the one in the figure below.

Handling User Input

Movement of the joystick is represented by the thumbstick ball moving around the canvas when the user drags it around. This is captured in the canvas .Dragged method.

This method does 3 simple things:

  1. If the ball is moving outside jsOuterRadius, the ball coordinates are recalculated and forced to stay in bounds.
  2. The current drag coordinates are recorded in jsPosX, jsPosY.
  3. Finally, the ball is moved to the current coordinates and the joystick values are displayed (see below).

Self Centering Behavior

Real joysticks come with self-centering and non self-centering thumbsticks. A 2-axis joystick can have a combination of the two types. The demo code has a checkbox that changes the centering behavior of both axes together (ie, they are always the same type), but can be easily modified to be whichever hybrid is needed.

When invoked, CenterJoystick resets the global tracking data to the center position and moves the thumbstick ball to that position.

The auto-centering implementation moves the joystick back to the middle as soon as the thumbstick is released, in the canvas .TouchUp method. In the alternative mode the joystick is centered manually using the on-screen button. Both call the same CenterJoystick procedure.

Using the Joystick Values

So once we have our virtual joystick working, how do we use the values for control?

One common application for a joystick is to control two different quantities, one per axis. In this example and demo code, the joystick controls a vehicle’s speed using the Y axis and steering direction using the X axis. The calcVelocityParam procedure calculates the control parameters from the joystick position information.

The speed (jsPctFullRange) is expressed as the percentage of full scale speed and is calculated as the percentage of the full scale joystick radius. Given the (x, y) position of the joystick, the speed radius is the hypotenuse of the triangle formed by the x and y values. If the joystick is in the lower half (below the center/origin of the joystick), the procedure adjusts the speed to be negative (ie reverse direction).

The turning angle (jsAngle) the angle of the joystick left or right of the Y (vertical). Vehicle steer direction is easiest expressed relative to the vertical y-axis (‘vehicle forward’) direction, with positive angles for right turn (+W) and negative angles for left turn (-W), illustrated in the figure below.

Given the (x, y) coordinates, the arctangent function is used give us the angle θ subtended by the line (0, 0) to (x, y). However, this is relative to the positive X axis rather than the vertical and so needs to be adjusted.

In the code, the angle W is calculated for the first quadrant (green Q1), as the 90° complement of θ. It is then adjusted depending on whether is lies above or below the X-axis (blue Q4), as shown in the figure above. Further, if the point actually sits on the negative side of the X-axis (yellow Q2, Q3) then it is simply the negative of the calculated angle (ie, 0-W).

Once we have these values, they can be compared to the previously calculated values (jsPctFullRangeOld and jsAngleOld) and, if they are changed, passed to the vehicle control system for action.

Other AI2 Joystick Implementations (variations on a theme)

App Inventor Joystick

Create A Bluetooth Joypad With App Inventor 2

MIT App Inventor: Joystick App

Joystick App for Arduino Tutorial – Part 1

Control your robot with your phone

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s