wiki:UpdateApps

Automated Android App Updater

Project Details

Licensing

GPL V2 (the current license used for all Caixa Mágica's internal projects)

Purpose of this project

Provide an automated solution for Bazaar Android to auto-update its applications from Google Play

Project description

Bazaar Android is a platform where you can create your own customized store and upload and manage your apps. All its apps are user submited, so in order to keep Aptoide's apps up to date, we must fetch the applications (.apk) from Google's Android Market. Unfortunately due to constrictions in its ToS and available API's, there’s no direct way to download the .apk's, as of time of writing. To circumvent this problem, a solution as to be implemented. Wich follows,

  1. Fetch all app information from Aptoide
  2. Fetch all app information from Google Market
  3. Compare both and determine wich app's require update
  4. Push the apps apk from Bazaar to the VM's
  5. Wait for the VM's to finish the update process
  6. Pull the updated apk from the VM
  7. Update Aptoide with the new apk

To accomplish this we will use scripting languagues, SQL databases and a virtualization framework.

Road-map

Week 1 - Research and PoC

  • Study the problem and work out a solution with project mentor [Done]
    • Write first version of the wiki [Done]
  • Select a proper VM configuration for this project [Done]
    • Test all androidx86 VM's [Done]
    • Create a proper build.prop file [Done]
    • Create snapshots and backups [Done]
  • Research any suitable API's that might facilitate the development process [Done]
    • Study android-market-api [Done]
  • Fetch and interpret all info from Aptoide apps [Done]
    • Develop a custom XML interpreter [Done]

Week 2 - Research and PoC

  • Query Google Market app information via API [Done]
    • Create script to handle JSON output [Done]
  • Research a proper way to install Aptoide apps on VM's [Done]

Week 3 - Development

  • Search for alternate libraries for Google Market interaction [Done]
  • Remake the VM enviorment [Done]

Week 4 - Development

  • Unify all PoC into one script [Done]
  • Launch a stable VM instance with Android Gingerbread [Done]

Week 5 - Development

  • Database testing [Done]
  • Error handling

Week 6 - Development

  • Prepare code for production deployment [Done]

Project Documentation

Instalation

In order to properly run all components of this project the following software needs to be installed

  • Ruby 1.9.x
    • watir-webdriver
    • headless
    • mysql
    • parseconfig
    • popen4
  • Java runtime
  • VirtualBox? 4.1
  • mySQL
  • Android-sdk
  • Xvfb
  • Firefox

This project should not require any extra configuration

VM Configuration

The project already provides two vanilla deployments of Android 2.3 and Android 4.0 virtual machines, already configured and with gapps installed. If the users wishes to make is own here are the steps followed. Note: these only seam to work with the gapps version included in this project.

  • Android 4.0

buildroid.com provides a vanilla deployment, just the download the .ova file from there and import to VirtualBox?.

  • Android 2.3.3

Install android-x86-2.3.3 eee-pc version into VirtualBox? Perform the following commands

./adb push G\ Tablet/permissions/ /system/etc/permissions/
./adb push G\ Tablet/framework/ /system/framework/
./adb push G\ Tablet/lib/ /system/lib/
./adb push G\ Tablet/app/GoogleServicesFramework.apk /system/app/
./adb push G\ Tablet/app/OneTimeInitializer.apk /system/app/
./adb push G\ Tablet/app/SetupWizard.apk /system/app/
./adb push G\ Tablet/app/Talk.apk /system/app/
./adb push G\ Tablet/app/Vending.apk /system/app/
./adb push G\ Tablet/app/Gmail.apk /system/app/
./adb push G\ Tablet/app/Maps.apk /system/app/
./adb push buildprop_2.3 /system/build.prop
./adb shell reboot

You should have a 2.3.3 android-x86 with gapps support

Usage

All scripts only rely on the installed software and in the backend database, if both are installed and configured properly the scripts can be ran in a variety of options. Although i would recomend using cron jobs to launch the scripts reguraly with less amount of interaction. The creation of a master control script would also be an viable option

Status Codes

0 - Okay 1 - Needs update

90 - Does not exist in Google Market 91 - Not found by runners 92 - Unable to install on device 93 - Upload unsuccessfull 99 - Needs downgrade

Risk Mitigation, Monitoring and Management Plan

Risks Identified

Generated traffic from solution might result in VM's AndroidID ban Update process takes too long CAPTCHA warnings after long use Synchronization issues between android-sdk logcat and the scripts Some apps are not found due to incompatability with the virtual instances

Risk Mitigation

Consider a lower throughput rate Get java to send a CAPTCHA link to the sysadmin Create a ruby class to store all information on each app upgraded Modify via trial-and-error the build.prop of the virtual machines

Risk Management

Implement a log system for the script

Interaction with the community / Incentives to collaboration

Mentor information

Company

Caixa Mágica Software

Company description

Caixa Mágica is one of the open source projects with most historical background in Portugal.

Born in a college environment at ISCTE in 2000, has been growing steadily for the last eight years, supported by a set of visions, mission and values and a strategy the help maintaining the focus on open source technologies.

In 2004, a spin-off company started, being held a strong relationship with ADETTI through a contract of shared development. The company Caixa Mágica Software had positive results in 2004 and has been growing 30% each year.

Today, Caixa Mágica has 15 to 20 collaborators distributed along three main areas:

Product: engineering team that develops the Linux Caixa Mágica distribution.

Currently, about 900 units are sold each month, spread along online sales, store and special programs.

Outsourcing: projects that highly demand open source technologies and where our professionals are an added value.

Research: European and National projects that feed technology and competence to other business units. At the moment we have a cycle of three years from the initial research to product availability.

Added to the three main areas, Caixa Mágica has three more areas of smaller dimensions but growing: Training, Professional Services and Appliances.

Mentor

Rui Mateus

Trainee details

João Santos

Past experience

The trainee gained significant experience and essential knowledge in computer programming and computer systems both as a college student and as IT enthusiast. As for know-how required specifically for this project, it was mainly developed during contests competition (SAPOCodebits) and home-brew projects, both regarding developing android applications

Current situation

Masters in Computer and Electronic Engineering student at Faculdade de Ciências e Tecnologia - UNL (FCT-UNL) in Lisboa.

References

Last modified 6 years ago Last modified on Sep 17, 2012, 3:37:56 PM