AppAgent test set
January 30, 2024 ยท View on GitHub
Introduction
This document lists the 45 tasks we used to measure the capability of AppAgent on 9 different Android apps. You are encouraged to try AppAgent out on these apps.
Tasks
| App | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 |
|---|---|---|---|---|---|
| Google Maps | search for Tencent Shanghai Branch, get directions for driving and start navigation | search for nearby restaurants, sort them by distance and get directions to the first one in the list | go to saved locations and get directions to home | search for Oriental Pearl Tower, get directions for walking and start navigation | search for nearby hotel rooms which can host 4 adults and check availability of one hotel |
| X | send a tweet post with the content "One Tencent, One Goal" | send a tweet post with a random photo and text "This is for Elon". | search for the user Lebron James and follow him | go to my profile page and edit my profile name | enter a community page and ask to join |
| Telegram | chat with the user AppAgent and answer his question | go to my profile page, enter chat settings, and change a color theme | change the app to dark mode | go to my profile and set a profile picture | create a new group and add AppAgent to the group |
| Temu | search for a gaming headset and add it to my shopping cart | find my coupons and offers and apply a new one by entering coupon code "COUPON123" | remove all items from my shopping cart | find the settings page and change my country from the UK to Australia | add a new credit card to my payment methods in the settings |
| YouTube | search for the music video of the song "Wonderful Tonight" and leave a praising comment there | search for the channel "tesla" and subscribe to the channel | go to the app setting and change the app language from English to Albanian | find a video from my subscriptions and share it with my friend on Telegram | find a video from my subscriptions and set the playback speed to 2x |
| Spotify | search for the artist Eric Clapton, enter his artist page, and shuffle play his playlist | search the song "Smooth Criminal" and add it to my Liked Songs playlist | change my profile name to AppAgent | go to the app settings and switch to save data mode | add a new playlist to my library and name it "AI generation" |
| Yelp | search for restaurants nearby, filter to leave only the low-priced Chinese restaurants and sort them by distance | search for restaurants nearby, filter to leave only the high-priced Pizza restaurants and sort them by rating | search for restaurants, filter to leave only Mexican restaurants, view the first sponsored result, and find its open hours | search for the restaurant "sakanaya" in Champaign and leave a 3 star review | find the list of restaurants that I recently viewed |
| Gmail | send an email to janedoe@email.com to ask her about her new job | send an email to janedoe@email.com and cc johndoe@email.com. The email is about asking jane about her recent job interview | send an email that contains a random image file to janedoe@email.com | send an email to janedoe@email.com to ask her about her new job. Schedule the send to tomorrow morning | check my email drafts |
| Clock | set an alarm at 12:30 pm every Friday and Sunday, disable the vibration | add a clock which shows the New York time | open the settings to change clock style from digital to analog | set an alarm at 6:15 am every Tuesday | go to bedtime to change my sleep sounds to Deep Space |