Pages

14 August 2012

Using java.awt.Robot to Create User Input Events

The motivation to use java.awt.Robot class is creating mouse-keyboard input events to test applications. Testing the application just using rough mouse and key event is extremely difficult. I think, there should be a handy framework to enhance and wrap java.awt.Robot class to support such following functionality:
  •  Taking ascii characters to map key events
  • Soft mouse movements instead of instant mouse location changes
  • Recording user actions to regenerate sequence of actions
  • Appropriate xml schema to save actions into xml file
  • windows, unix and mac native implementations to manage os specific actions that are not supported by java.awt.Robot
As a kickstart, I tried to explicitly show off setAutoDelay, keyPress, keyRelase, mouseWheel, mouseMove methods usages. Comments over code blocks are covering the all following code target action. Obviously, a single tab key, costs 2 lines of code; this is too much for a single key if you are developing a great application project with hundreds of text fields.

Following class only works on Windows that act as follows:
  1. show desktop 
  2. open default browser to go "www.google.com"
  3. wait
  4. type "0guzhan.blogspot.com mensup"
  5. then tab to search button
  6. enter search button
  7. then create mouse wheel and mouse move events, forever...
The created events are at OS level; thus microsoft office communicator perceives them as user's real actions and never go to its inactive state :)


import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;
import java.io.IOException;

public class Mensup {

 public static void main(String[] args) {
  try {
   final Robot robot = new Robot();
   // set teh default delay time 
   // between any action on this robot instance
   robot.setAutoDelay(500);
   
   // show desktop on windows
   robot.keyPress(KeyEvent.VK_WINDOWS);
   robot.keyPress(KeyEvent.VK_D);
   robot.keyRelease(KeyEvent.VK_D);
   robot.keyRelease(KeyEvent.VK_WINDOWS);
   
   try {
    // open default browser of windows NT to go "www.google.com"
    Runtime.getRuntime().exec("RunDLL32.EXE shell32.dll,ShellExec_RunDLL http://www.google.com");
    // wait for browser startup time
    robot.delay(5000);
   } catch (IOException e) {
    e.printStackTrace();
   }
   
   // alt-tab to select browser
   robot.keyPress(KeyEvent.VK_ALT);
   robot.keyPress(KeyEvent.VK_TAB);
   robot.keyPress(KeyEvent.VK_ALT);
   robot.keyPress(KeyEvent.VK_TAB);
   robot.keyRelease(KeyEvent.VK_ALT);
   robot.keyRelease(KeyEvent.VK_TAB);
   
   // craete inputs for each character in text array
   for (int character : text) {
    robot.keyPress(character);
    robot.keyRelease(character);
   }
   
   // tab to search button
   robot.keyPress(KeyEvent.VK_TAB);
   robot.keyRelease(KeyEvent.VK_TAB);
   // enter search button
   robot.keyPress(KeyEvent.VK_ENTER);
   robot.keyRelease(KeyEvent.VK_ENTER);
   
   // after search on google
   // play on the opened window with scroll
   while (true) {
    robot.mouseWheel(20);
    robot.mouseWheel(-20);
    robot.mouseMove(50, 50);
    robot.mouseMove(250, 250);
   } 
  } catch (AWTException e) {
   e.printStackTrace();
  }
 }
 
 static final int[] text = new int[] {KeyEvent.VK_0,
  KeyEvent.VK_G,
  KeyEvent.VK_U,
  KeyEvent.VK_Z,
  KeyEvent.VK_H,
  KeyEvent.VK_A,
  KeyEvent.VK_N,
  KeyEvent.VK_PERIOD,
  KeyEvent.VK_B,
  KeyEvent.VK_L,
  KeyEvent.VK_O,
  KeyEvent.VK_G,
  KeyEvent.VK_S,
  KeyEvent.VK_P,
  KeyEvent.VK_O,
  KeyEvent.VK_T,
  KeyEvent.VK_PERIOD,
  KeyEvent.VK_C,
  KeyEvent.VK_O,
  KeyEvent.VK_M,
  KeyEvent.VK_SPACE,
  KeyEvent.VK_R,
  KeyEvent.VK_O,
  KeyEvent.VK_B,
  KeyEvent.VK_O,
  KeyEvent.VK_T};
}

Take care...

5 comments:

  1. This article is very nice i like it this is paragraph is so nice its very informative.
    Advertising | Advertising agency

    ReplyDelete
  2. keypress'de türkçe karakterler nasıl çıkıyor? daha doğrusu küçük i için sürekli ı basıyor robot.

    ReplyDelete
  3. KeyEvent sınıfında var olan, karakterler dışında input yaratılamıyor. Denediğimde, "Invalid key" hatası atıyor. java.awt.Robot kullanmak yerine JNA' yı kullanmanı tavsiye ederim. iyi günler

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Cevabın için teşekkürler ama baya bir uğraştıktan sorunu şu şekilde hallettim:
    Sorun: KeyEvent.VK_I türkçe klavyede 'ı' tuşu basıyor, 'i' yazamıyorduk.
    Çözüm: Bunun için alt tuş kombinasyonu kullanabiliriz. Bu durum için alt + 105(numpad) tuş kombinasyonu 'i' basmaktadır.

    robot.keyPress(KeyEvent.VK_ALT);
    keyPressAndRelease(KeyEvent.VK_NUMPAD1);
    keyPressAndRelease(KeyEvent.VK_NUMPAD0);
    keyPressAndRelease(KeyEvent.VK_NUMPAD5);
    robot.keyRelease(KeyEvent.VK_ALT);

    void keyPressAndRelease(int key) {
    robot.keyPress(key);
    robot.keyRelease(key);
    }

    Türkçe karakterler 'as a classic' her yerde başımıza bela. Türkçe sorunlara yabancı kaynaklardan çözüm bulmak da zor haliyle. İhtiyacı olan biri olabilir düşüncesiyle bu soruna olası bir çözümü yazmış bulundum.

    ReplyDelete

Thx for reading! Comments are appreciated...