Block based ota

Block based ota DEFAULT

Block-based OTA

You can enable new devices running Android 5.0 to initiate block-based OTA upgrades. OTA is a mechanism for device operators to remotely update part of a device's system.

  • Android 5.0 and subsequent versions use block device OTA upgrades to ensure that each device uses the same exact partition. Block OTA processes the entire partition as a file and calculates a single binary patch, instead of comparing individual files and calculating the binary patch, to ensure that the generated partition saves the precise destination. This allows the device system image to achieve the same state through fastboot or OTA.

  • Android4.4 and previous versions use file OTA update. File OTA ensures that the device contains the same file content, permissions and mode, but based on the update method, it allows metadata such as timestamp and underlying storage distribution to distinguish devices.

Since block OTA ensures that every device uses the same partition, it allows the use of DM authentication to encrypt the system partition. We will talk about dm verification in the "Verify Boot" section later.

Note: Before using dm authentication, you must have an available block OTA system.


For devices with Android 5.0 or subsequent versions, the block OTA update is used in the factory ROM. In order to generate a block-based OTA for subsequent updates, pass the –block parameter to ota_from_target_files.

For Android4.4 or previous versions, use file OTA update. Although it is possible to send a complete Android5.0 or later version of the block OTA to convert the device, he needs to send a complete OTA, which is much larger than an incremental OTA.

Because dm verification requires bootloader support in new devices of Android 5.0 or subsequent versions, for existing devices, you cannot use dm verification.

Developers working on the Android OTA system (recovery mirroring and OTA generation script) can keep changes by subscribing to the [email protected] mailing list.

File VS Block OTA

During a file-based OTA update, Android tries to change the contents of the system partition located at the file system layer (file-based files). Update does not guarantee to write files in a consistent order, have a consistent last modification time and super block, or even place the block in the same location on the block device. For this reason, file-based OTA will fail on a device with dm authentication enabled; after an OTA attempt, the device will not start up.

During the block-based OTA update, Android provides the difference between the two block images of the device (rather than the collection of two sets of files). The update program uses the following method to check the corresponding build service at the block device layer (located under the file system):

  • Full update. Copying the entire system image is simple, and generating patches is also very simple. But generating a large image can make making and applying patches very expensive.

  • Incremental update. Using the binary diff tool to generate a smaller image and make it easy to generate patch applications, but it is memory-intensive when generating patches.
    Note: adb fastboot regards exactly the same bits in the device as a complete OTA, so flashing is compatible with block OTA.

Update an unimproved system

For devices running Android 5.0 with unmodified system partitions, the download and installation of block OTA is the same as file OTA. However, OTA updates may contain one or more of the following differences:

  • Download size. The full block OTA update is about the same size as the full file OTA update, but the incremental update may be as large as several megabytes.

    View Image
    compares the size of Nexus 6 OTA for Android 5.0 and Android 5.1 releases

  • In general, incremental block OTA updates are larger than incremental file OTA updates because:

    • Data preservation. Block-based OTA saves more data (file metadata, dm verification data, ext4 distribution, etc.) than file-based OTA.
    • The calculation algorithm is different. In an OTA update of a file, if the file path is the same in the two build versions, the OTA package does not contain data for that file. In a block OTA update, the decision to make little or no change in a file depends on the quality of the patch calculation algorithm and the file data layout in the source and target systems.
  • Sensitivity of fault flash and RAM. If a file is damaged, if it does not touch the damaged file, the file OTA will still proceed; but if the block OTA detects damage on the system partition, the block OTA will fail.

Updated and improved system

For devices with improved system partitions running Android 5.0:

  • OTA update of incremental block failed. The system partition may be modified by malware during adb remounting. File OTA tolerates some changes to the partition, such as adding additional files that are not source code or build targets. However, block OTA update does not tolerate any addition of partitions, so users need to install a full OTA to cover the system partition or burn in a new system image to enable future OTA updates.

  • Trying to change the improved file caused the update to fail. For file and block OTA updates, if the OTA tries to modify a file that has been improved, the OTA update will fail.

  • Attempting to access the improved file may generate an error (only dm verification). For file and block OTA updates, if dm verification is enabled and OTA tries to access part of the improved file system, OTA will generate an error.

#!/usr/bin/env python## Copyright (C) 2008 The Android Open Source Project## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License."""Given a target-files zipfile, produces an OTA package that installs that build.An incremental OTA is produced if -i is given, otherwise a full OTA is produced.Usage: ota_from_target_files [options] input_target_files output_ota_packageCommon options that apply to both of non-A/B and A/B OTAs --downgrade Intentionally generate an incremental OTA that updates from a newer build to an older one (e.g. downgrading from P preview back to O MR1). "ota-downgrade=yes" will be set in the package metadata file. A data wipe will always be enforced when using this flag, so "ota-wipe=yes" will also be included in the metadata file. The update-binary in the source build will be used in the OTA package, unless --binary flag is specified. Please also check the comment for --override_timestamp below. -i (--incremental_from) <file> Generate an incremental OTA using the given target-files zip as the starting build. -k (--package_key) <key> Key to use to sign the package (default is the value of default_system_dev_certificate from the input target-files's META/misc_info.txt, or "build/make/target/product/security/testkey" if that value is not specified). For incremental OTAs, the default value is based on the source target-file, not the target build. --override_timestamp Intentionally generate an incremental OTA that updates from a newer build to an older one (based on timestamp comparison), by setting the downgrade flag in the package metadata. This differs from --downgrade flag, as we don't enforce a data wipe with this flag. Because we know for sure this is NOT an actual downgrade case, but two builds happen to be cut in a reverse order (e.g. from two branches). A legit use case is that we cut a new build C (after having A and B), but want to enfore an update path of A -> C -> B. Specifying --downgrade may not help since that would enforce a data wipe for C -> B update. We used to set a fake timestamp in the package metadata for this flow. But now we consolidate the two cases (i.e. an actual downgrade, or a downgrade based on timestamp) with the same "ota-downgrade=yes" flag, with the difference being whether "ota-wipe=yes" is set. --wipe_user_data Generate an OTA package that will wipe the user data partition when installed. --retrofit_dynamic_partitions Generates an OTA package that updates a device to support dynamic partitions (default False). This flag is implied when generating an incremental OTA where the base build does not support dynamic partitions but the target build does. For A/B, when this flag is set, --skip_postinstall is implied. --skip_compatibility_check Skip checking compatibility of the input target files package. --output_metadata_path Write a copy of the metadata to a separate file. Therefore, users can read the post build fingerprint without extracting the OTA package. --force_non_ab This flag can only be set on an A/B device that also supports non-A/B updates. Implies --two_step. If set, generate that non-A/B update package. If not set, generates A/B package for A/B device and non-A/B package for non-A/B device. -o (--oem_settings) <main_file[,additional_files...]> Comma separated list of files used to specify the expected OEM-specific properties on the OEM partition of the intended device. Multiple expected values can be used by providing multiple files. Only the first dict will be used to compute fingerprint, while the rest will be used to assert OEM-specific properties.Non-A/B OTA specific options -b (--binary) <file> Use the given binary as the update-binary in the output package, instead of the binary in the build's target_files. Use for development only. --block Generate a block-based OTA for non-A/B device. We have deprecated the support for file-based OTA since O. Block-based OTA will be used by default for all non-A/B devices. Keeping this flag here to not break existing callers. -e (--extra_script) <file> Insert the contents of file at the end of the update script. --full_bootloader Similar to --full_radio. When generating an incremental OTA, always include a full copy of bootloader image. --full_radio When generating an incremental OTA, always include a full copy of radio image. This option is only meaningful when -i is specified, because a full radio is always included in a full OTA if applicable. --log_diff <file> Generate a log file that shows the differences in the source and target builds for an incremental package. This option is only meaningful when -i is specified. --oem_no_mount For devices with OEM-specific properties but without an OEM partition, do not mount the OEM partition in the updater-script. This should be very rarely used, since it's expected to have a dedicated OEM partition for OEM-specific properties. Only meaningful when -o is specified. --stash_threshold <float> Specify the threshold that will be used to compute the maximum allowed stash size (defaults to 0.8). -t (--worker_threads) <int> Specify the number of worker-threads that will be used when generating patches for incremental updates (defaults to 3). --verify Verify the checksums of the updated system and vendor (if any) partitions. Non-A/B incremental OTAs only. -2 (--two_step) Generate a 'two-step' OTA package, where recovery is updated first, so that any changes made to the system partition are done using the new recovery (new kernel, etc.).A/B OTA specific options --disable_fec_computation Disable the on device FEC data computation for incremental updates. --include_secondary Additionally include the payload for secondary slot images (default: False). Only meaningful when generating A/B OTAs. By default, an A/B OTA package doesn't contain the images for the secondary slot (e.g. system_other.img). Specifying this flag allows generating a separate payload that will install secondary slot images. Such a package needs to be applied in a two-stage manner, with a reboot in-between. During the first stage, the updater applies the primary payload only. Upon finishing, it reboots the device into the newly updated slot. It then continues to install the secondary payload to the inactive slot, but without switching the active slot at the end (needs the matching support in update_engine, i.e. SWITCH_SLOT_ON_REBOOT flag). Due to the special install procedure, the secondary payload will be always generated as a full payload. --payload_signer <signer> Specify the signer when signing the payload and metadata for A/B OTAs. By default (i.e. without this flag), it calls 'openssl pkeyutl' to sign with the package private key. If the private key cannot be accessed directly, a payload signer that knows how to do that should be specified. The signer will be supplied with "-inkey <path_to_key>", "-in <input_file>" and "-out <output_file>" parameters. --payload_signer_args <args> Specify the arguments needed for payload signer. --payload_signer_maximum_signature_size <signature_size> The maximum signature size (in bytes) that would be generated by the given payload signer. Only meaningful when custom payload signer is specified via '--payload_signer'. If the signer uses a RSA key, this should be the number of bytes to represent the modulus. If it uses an EC key, this is the size of a DER-encoded ECDSA signature. --payload_signer_key_size <key_size> Deprecated. Use the '--payload_signer_maximum_signature_size' instead. --boot_variable_file <path> A file that contains the possible values of ro.boot.* properties. It's used to calculate the possible runtime fingerprints when some ro.product.* properties are overridden by the 'import' statement. The file expects one property per line, and each line has the following format: 'prop_name=value1,value2'. e.g. 'ro.boot.product.sku=std,pro' --skip_postinstall Skip the postinstall hooks when generating an A/B OTA package (default: False). Note that this discards ALL the hooks, including non-optional ones. Should only be used if caller knows it's safe to do so (e.g. all the postinstall work is to dexopt apps and a data wipe will happen immediately after). Only meaningful when generating A/B OTAs. --partial "<PARTITION> [<PARTITION>[...]]" Generate partial updates, overriding ab_partitions list with the given list. --custom_image <custom_partition=custom_image> Use the specified custom_image to update custom_partition when generating an A/B OTA package. e.g. "--custom_image oem=oem.img --custom_image cus=cus_test.img" --disable_vabc Disable Virtual A/B Compression, for builds that have compression enabled by default. --vabc_downgrade Don't disable Virtual A/B Compression for downgrading OTAs. For VABC downgrades, we must finish merging before doing data wipe, and since data wipe is required for downgrading OTA, this might cause long wait time in recovery. --enable_vabc_xor Enable the VABC xor feature. Will reduce space requirements for OTA --force_minor_version Override the update_engine minor version for delta generation."""from __future__ import print_functionimport loggingimport multiprocessingimport osimport os.pathimport reimport shleximport shutilimport structimport subprocessimport sysimport zipfileimport care_map_pb2import commonimport ota_utilsfrom ota_utils import (UNZIP_PATTERN, FinalizeMetadata, GetPackageMetadata, PropertyFiles, SECURITY_PATCH_LEVEL_PROP_NAME, GetZipEntryOffset)import target_files_difffrom check_target_files_vintf import CheckVintfIfTrebleEnabledfrom non_ab_ota import GenerateNonAbOtaPackageif sys.hexversion < 0x02070000: print("Python 2.7 or newer is required.", file=sys.stderr) sys.exit(1)logger = logging.getLogger(__name__)OPTIONS = ota_utils.OPTIONSOPTIONS.verify = FalseOPTIONS.patch_threshold = 0.95OPTIONS.wipe_user_data = FalseOPTIONS.extra_script = NoneOPTIONS.worker_threads = multiprocessing.cpu_count() // 2if OPTIONS.worker_threads == 0: OPTIONS.worker_threads = 1OPTIONS.two_step = FalseOPTIONS.include_secondary = FalseOPTIONS.block_based = TrueOPTIONS.updater_binary = NoneOPTIONS.oem_dicts = NoneOPTIONS.oem_source = NoneOPTIONS.oem_no_mount = FalseOPTIONS.full_radio = FalseOPTIONS.full_bootloader = False# Stash size cannot exceed cache_size * threshold.OPTIONS.cache_size = NoneOPTIONS.stash_threshold = 0.8OPTIONS.log_diff = NoneOPTIONS.payload_signer = NoneOPTIONS.payload_signer_args = []OPTIONS.payload_signer_maximum_signature_size = NoneOPTIONS.extracted_input = NoneOPTIONS.skip_postinstall = FalseOPTIONS.skip_compatibility_check = FalseOPTIONS.disable_fec_computation = FalseOPTIONS.disable_verity_computation = FalseOPTIONS.partial = NoneOPTIONS.custom_images = {}OPTIONS.disable_vabc = FalseOPTIONS.spl_downgrade = FalseOPTIONS.vabc_downgrade = FalseOPTIONS.enable_vabc_xor = TrueOPTIONS.force_minor_version = NonePOSTINSTALL_CONFIG = 'META/postinstall_config.txt'DYNAMIC_PARTITION_INFO = 'META/dynamic_partitions_info.txt'AB_PARTITIONS = 'META/ab_partitions.txt'# Files to be unzipped for target diffing purpose.TARGET_DIFFING_UNZIP_PATTERN = ['BOOT', 'RECOVERY', 'SYSTEM/*', 'VENDOR/*', 'PRODUCT/*', 'SYSTEM_EXT/*', 'ODM/*', 'VENDOR_DLKM/*', 'ODM_DLKM/*']RETROFIT_DAP_UNZIP_PATTERN = ['OTA/super_*.img', AB_PARTITIONS]# Images to be excluded from secondary payload. We essentially only keep# 'system_other' and bootloader partitions.SECONDARY_PAYLOAD_SKIPPED_IMAGES = [ 'boot', 'dtbo', 'modem', 'odm', 'odm_dlkm', 'product', 'radio', 'recovery', 'system_ext', 'vbmeta', 'vbmeta_system', 'vbmeta_vendor', 'vendor', 'vendor_boot']class PayloadSigner(object): """A class that wraps the payload signing works. When generating a Payload, hashes of the payload and metadata files will be signed with the device key, either by calling an external payload signer or by calling openssl with the package key. This class provides a unified interface, so that callers can just call PayloadSigner.Sign(). If an external payload signer has been specified (OPTIONS.payload_signer), it calls the signer with the provided args (OPTIONS.payload_signer_args). Note that the signing key should be provided as part of the payload_signer_args. Otherwise without an external signer, it uses the package key (OPTIONS.package_key) and calls openssl for the signing works. """ def __init__(self): if OPTIONS.payload_signer is None: # Prepare the payload signing key. private_key = OPTIONS.package_key + OPTIONS.private_key_suffix pw = OPTIONS.key_passwords[OPTIONS.package_key] cmd = ["openssl", "pkcs8", "-in", private_key, "-inform", "DER"] cmd.extend(["-passin", "pass:" + pw] if pw else ["-nocrypt"]) signing_key = common.MakeTempFile(prefix="key-", suffix=".key") cmd.extend(["-out", signing_key]) common.RunAndCheckOutput(cmd, verbose=False) self.signer = "openssl" self.signer_args = ["pkeyutl", "-sign", "-inkey", signing_key, "-pkeyopt", "digest:sha256"] self.maximum_signature_size = self._GetMaximumSignatureSizeInBytes( signing_key) else: self.signer = OPTIONS.payload_signer self.signer_args = OPTIONS.payload_signer_args if OPTIONS.payload_signer_maximum_signature_size: self.maximum_signature_size = int( OPTIONS.payload_signer_maximum_signature_size) else: # The legacy config uses RSA2048 keys. logger.warning("The maximum signature size for payload signer is not" " set, default to 256 bytes.") self.maximum_signature_size = 256 @staticmethod def _GetMaximumSignatureSizeInBytes(signing_key): out_signature_size_file = common.MakeTempFile("signature_size") cmd = ["delta_generator", "--out_maximum_signature_size_file={}".format( out_signature_size_file), "--private_key={}".format(signing_key)] common.RunAndCheckOutput(cmd) with open(out_signature_size_file) as f: signature_size ="%s outputs the maximum signature size: %s", cmd[0], signature_size) return int(signature_size) def Sign(self, in_file): """Signs the given input file. Returns the output filename.""" out_file = common.MakeTempFile(prefix="signed-", suffix=".bin") cmd = [self.signer] + self.signer_args + ['-in', in_file, '-out', out_file] common.RunAndCheckOutput(cmd) return out_fileclass Payload(object): """Manages the creation and the signing of an A/B OTA Payload.""" PAYLOAD_BIN = 'payload.bin' PAYLOAD_PROPERTIES_TXT = 'payload_properties.txt' SECONDARY_PAYLOAD_BIN = 'secondary/payload.bin' SECONDARY_PAYLOAD_PROPERTIES_TXT = 'secondary/payload_properties.txt' def __init__(self, secondary=False): """Initializes a Payload instance. Args: secondary: Whether it's generating a secondary payload (default: False). """ self.payload_file = None self.payload_properties = None self.secondary = secondary def _Run(self, cmd): # pylint: disable=no-self-use # Don't pipe (buffer) the output if verbose is set. Let # brillo_update_payload write to stdout/stderr directly, so its progress can # be monitored. if OPTIONS.verbose: common.RunAndCheckOutput(cmd, stdout=None, stderr=None) else: common.RunAndCheckOutput(cmd) def Generate(self, target_file, source_file=None, additional_args=None): """Generates a payload from the given target-files zip(s). Args: target_file: The filename of the target build target-files zip. source_file: The filename of the source build target-files zip; or None if generating a full OTA. additional_args: A list of additional args that should be passed to brillo_update_payload script; or None. """ if additional_args is None: additional_args = [] payload_file = common.MakeTempFile(prefix="payload-", suffix=".bin") cmd = ["brillo_update_payload", "generate", "--payload", payload_file, "--target_image", target_file] if source_file is not None: cmd.extend(["--source_image", source_file]) if OPTIONS.disable_fec_computation: cmd.extend(["--disable_fec_computation", "true"]) if OPTIONS.disable_verity_computation: cmd.extend(["--disable_verity_computation", "true"]) cmd.extend(additional_args) self._Run(cmd) self.payload_file = payload_file self.payload_properties = None def Sign(self, payload_signer): """Generates and signs the hashes of the payload and metadata. Args: payload_signer: A PayloadSigner() instance that serves the signing work. Raises: AssertionError: On any failure when calling brillo_update_payload script. """ assert isinstance(payload_signer, PayloadSigner) # 1. Generate hashes of the payload and metadata files. payload_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin") metadata_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin") cmd = ["brillo_update_payload", "hash", "--unsigned_payload", self.payload_file, "--signature_size", str(payload_signer.maximum_signature_size), "--metadata_hash_file", metadata_sig_file, "--payload_hash_file", payload_sig_file] self._Run(cmd) # 2. Sign the hashes. signed_payload_sig_file = payload_signer.Sign(payload_sig_file) signed_metadata_sig_file = payload_signer.Sign(metadata_sig_file) # 3. Insert the signatures back into the payload file. signed_payload_file = common.MakeTempFile(prefix="signed-payload-", suffix=".bin") cmd = ["brillo_update_payload", "sign", "--unsigned_payload", self.payload_file, "--payload", signed_payload_file, "--signature_size", str(payload_signer.maximum_signature_size), "--metadata_signature_file", signed_metadata_sig_file, "--payload_signature_file", signed_payload_sig_file] self._Run(cmd) # 4. Dump the signed payload properties. properties_file = common.MakeTempFile(prefix="payload-properties-", suffix=".txt") cmd = ["brillo_update_payload", "properties", "--payload", signed_payload_file, "--properties_file", properties_file] self._Run(cmd) if self.secondary: with open(properties_file, "a") as f: f.write("SWITCH_SLOT_ON_REBOOT=0\n") if OPTIONS.wipe_user_data: with open(properties_file, "a") as f: f.write("POWERWASH=1\n") self.payload_file = signed_payload_file self.payload_properties = properties_file def WriteToZip(self, output_zip): """Writes the payload to the given zip. Args: output_zip: The output ZipFile instance. """ assert self.payload_file is not None assert self.payload_properties is not None if self.secondary: payload_arcname = Payload.SECONDARY_PAYLOAD_BIN payload_properties_arcname = Payload.SECONDARY_PAYLOAD_PROPERTIES_TXT else: payload_arcname = Payload.PAYLOAD_BIN payload_properties_arcname = Payload.PAYLOAD_PROPERTIES_TXT # Add the signed payload file and properties into the zip. In order to # support streaming, we pack them as ZIP_STORED. So these entries can be # read directly with the offset and length pairs. common.ZipWrite(output_zip, self.payload_file, arcname=payload_arcname, compress_type=zipfile.ZIP_STORED) common.ZipWrite(output_zip, self.payload_properties, arcname=payload_properties_arcname, compress_type=zipfile.ZIP_STORED)def _LoadOemDicts(oem_source): """Returns the list of loaded OEM properties dict.""" if not oem_source: return None oem_dicts = [] for oem_file in oem_source: with open(oem_file) as fp: oem_dicts.append(common.LoadDictionaryFromLines(fp.readlines())) return oem_dictsclass StreamingPropertyFiles(PropertyFiles): """A subclass for computing the property-files for streaming A/B OTAs.""" def __init__(self): super(StreamingPropertyFiles, self).__init__() = 'ota-streaming-property-files' self.required = ( # payload.bin and payload_properties.txt must exist. 'payload.bin', 'payload_properties.txt', ) self.optional = ( # apex_info.pb isn't directly used in the update flow 'apex_info.pb', # care_map is available only if dm-verity is enabled. 'care_map.pb', 'care_map.txt', # is available only if target supports Treble. '', )class AbOtaPropertyFiles(StreamingPropertyFiles): """The property-files for A/B OTA that includes payload_metadata.bin info. Since P, we expose one more token (aka property-file), in addition to the ones for streaming A/B OTA, for a virtual entry of 'payload_metadata.bin'. 'payload_metadata.bin' is the header part of a payload ('payload.bin'), which doesn't exist as a separate ZIP entry, but can be used to verify if the payload can be applied on the given device. For backward compatibility, we keep both of the 'ota-streaming-property-files' and the newly added 'ota-property-files' in P. The new token will only be available in 'ota-property-files'. """ def __init__(self): super(AbOtaPropertyFiles, self).__init__() = 'ota-property-files' def _GetPrecomputed(self, input_zip): offset, size = self._GetPayloadMetadataOffsetAndSize(input_zip) return ['payload_metadata.bin:{}:{}'.format(offset, size)] @staticmethod def _GetPayloadMetadataOffsetAndSize(input_zip): """Computes the offset and size of the payload metadata for a given package. (From system/update_engine/update_metadata.proto) A delta update file contains all the deltas needed to update a system from one specific version to another specific version. The update format is represented by this struct pseudocode: struct delta_update_file { char magic[4] = "CrAU"; uint64 file_format_version; uint64 manifest_size; // Size of protobuf DeltaArchiveManifest // Only present if format_version > 1: uint32 metadata_signature_size; // The Bzip2 compressed DeltaArchiveManifest char manifest[metadata_signature_size]; // The signature of the metadata (from the beginning of the payload up to // this location, not including the signature itself). This is a // serialized Signatures message. char medatada_signature_message[metadata_signature_size]; // Data blobs for files, no specific format. The specific offset // and length of each data blob is recorded in the DeltaArchiveManifest. struct { char data[]; } blobs[]; // These two are not signed: uint64 payload_signatures_message_size; char payload_signatures_message[]; }; 'payload-metadata.bin' contains all the bytes from the beginning of the payload, till the end of 'medatada_signature_message'. """ payload_info = input_zip.getinfo('payload.bin') (payload_offset, payload_size) = GetZipEntryOffset(input_zip, payload_info) # Read the underlying raw zipfile at specified offset payload_fp = input_zip.fp header_bin = # network byte order (big-endian) header = struct.unpack("!IQQL", header_bin) # 'CrAU' magic = header[0] assert magic == 0x43724155, "Invalid magic: {:x}, computed offset {}" \ .format(magic, payload_offset) manifest_size = header[2] metadata_signature_size = header[3] metadata_total = 24 + manifest_size + metadata_signature_size assert metadata_total < payload_size return (payload_offset, metadata_total)def UpdatesInfoForSpecialUpdates(content, partitions_filter, delete_keys=None): """ Updates info file for secondary payload generation, partial update, etc. Scan each line in the info file, and remove the unwanted partitions from the dynamic partition list in the related properties. e.g. "super_google_dynamic_partitions_partition_list=system vendor product" will become "super_google_dynamic_partitions_partition_list=system". Args: content: The content of the input info file. e.g. misc_info.txt. partitions_filter: A function to filter the desired partitions from a given list delete_keys: A list of keys to delete in the info file Returns: A string of the updated info content. """ output_list = [] # The suffix in partition_list variables that follows the name of the # partition group. list_suffix = 'partition_list' for line in content.splitlines(): if line.startswith('#') or '=' not in line: output_list.append(line) continue key, value = line.strip().split('=', 1) if delete_keys and key in delete_keys: pass elif key.endswith(list_suffix): partitions = value.split() # TODO for partial update, partitions in the same group must be all # updated or all omitted partitions = filter(partitions_filter, partitions) output_list.append('{}={}'.format(key, ' '.join(partitions))) else: output_list.append(line) return '\n'.join(output_list)def GetTargetFilesZipForSecondaryImages(input_file, skip_postinstall=False): """Returns a file for generating secondary payload. Although the original already contains secondary slot images (i.e. IMAGES/system_other.img), we need to rename the files to the ones without _other suffix. Note that we cannot instead modify the names in META/ab_partitions.txt, because there are no matching partitions on device. For the partitions that don't have secondary images, the ones for primary slot will be used. This is to ensure that we always have valid boot, vbmeta, bootloader images in the inactive slot. Args: input_file: The input file. skip_postinstall: Whether to skip copying the postinstall config file. Returns: The filename of the for generating secondary payload. """ def GetInfoForSecondaryImages(info_file): """Updates info file for secondary payload generation.""" with open(info_file) as f: content = # Remove virtual_ab flag from secondary payload so that OTA client # don't use snapshots for secondary update delete_keys = ['virtual_ab', "virtual_ab_retrofit"] return UpdatesInfoForSpecialUpdates( content, lambda p: p not in SECONDARY_PAYLOAD_SKIPPED_IMAGES, delete_keys)
  1. Colt m4 serial numbers
  2. C5 corvette hardtop conversion
  3. Cadillac ctsv price
  4. Extra 300 unemployment pa update

An OTA update has arrived, but you cannot install it because your Android device is rooted? Well, that’s a known downside of rooting Android. In this guide, you will learn how to install OTA updates on rooted Android devices using Magisk. By using the method provided in this guide, you will also be able to retain root and TWRP after installing the OTA update. Before we get any further, let us take a quick look at what OTA updates are and how they work.

What is an OTA Update?

An OTA Update a.k.a an Over-the-Air Update is a wireless method of receiving and installing updates to an Android device’s software.

It is remotely pushed by the device manufacturer (Also known as the ‘OEM’) and is the easiest and most effective way to update an Android device’s software to the latest available version. Once an OTA update arrives, it is either automatically installed or prompts the user to manually download and install it.

So, how does one install an OTA update on a rooted Android device? You will know more about it in the next section. Read ahead.

Why You Can’t Install OTA Updates on Rooted Android Devices Automatically?

Rooting an Android device is great, it gives the user tremendous control over the device’s software and allows making any sort of modifications to it. However, it comes with its own drawbacks. The biggest one is losing the ability to take automatic OTA (over-the-air) updates. And users (Generally who are new to the scene) often tend to overlook this factor when they root their Android device.

Why you can't install OTA updates on rooted Android devices

In Android 5.0 and above, Google introduced the concept of block-based OTAs. This new mechanism does certain pre-OTA block checks to verify the integrity of the device’s software. In layman terms, once the installation begins, the device partitions (like system, boot, vendor, dtbo, etc) are checked for modifications like Root, TWRP, etc. And if detected, the OTA update will fail to install.

Magisk Root, A/B Partitions, and OTA Updates

But thanks to Magisk, you can easily install OTA updates on rooted Android devices without losing root. For those who don’t know, Magisk is a Universal Systemless Interface that allows you to make modifications to your Android device’s software systemless-ly, that is, without altering the /system partition.

Not just that, it has also developed to be the most prominent modern-day rooting solution for Android using MagiskSU. Want to know more about how it came into existence? Then there’s no better way but to hear out the story directly from its creator – topjohnwu.

The developer devised two different methods to install Magisk and root Android devices:

  1. The first method follows the traditional process of installing a custom recovery (TWRP, OrangeFox, etc) and then using it to flash the Magisk installer zip (root package). But since you will install a custom recovery during this method, it makes it even harder to take OTA updates. So, to regain the ability to take automatic OTA updates or install them manually, you would require to first uninstall/remove root and other modifications applied to the software. And further, restore the stock Android recovery on your device.
  2. The second method was introduced when Google first released the Google Pixel devices in 2016. These devices came with dual A/B partition system to support seamless updates and removed the recovery partition from the devices completely. So, the first method failed rigorously. This new method involves patching the stock boot image using Magisk and then flashing the patched boot image via Fastboot.

The best part about rooting your Android device with Magisk is that it gives you the ability to install OTA updates easily. So, follow the instructions below on how to do it.

How to Install OTA Updates on Rooted Android Devices?

Now, since most new OEM Android devices come with the A/B partition scheme, OTA updates are seamlessly installed to the inactive slot. When you root using Magisk, it is only installed to the currently active slot of your Android device. And thus, the inactive slot/partitions stay untouched.

When an OTA arrives, you can simply restore the stock boot image from within Magisk Manager, then install the OTA update to the inactive slot. And finally, right before you reboot your phone, you can install Magisk to the inactive slot to preserve root after the update.

Still, sounds a bit confusing? Don’t worry, the instructions below will take you through the complete process in detail. For your convenience, we have split the instructions into 5 different steps so that you can understand exactly what you’re doing.

Step 1: Disable Automatic System Updates in Settings

In Android 8.0 Oreo (and above), Google introduced a new automatic updates policy that downloads and installs OTA updates as soon as they are available.

Although this is great for an average Android user, but not for those who have rooted their devices. In order to update your rooted Android phone, you must first prevent the system from automatically installing updates without your acknowledgment.

Simply follow the steps below to disable Automatic System Updates in Android:

  1. Go to the Settings menu on your Android device.
  2. Navigate through the settings and select the About phone menu.
  3. Scroll down to the bottom and find the Build number section.
    • Note: In the case of some OEM devices, the Build Number section could be nested further in the Settings menu. For example, in Samsung Galaxy devices, you will need to go to Settings → About phone → Software information.
  4. Now, continuously tap on the ‘Build number’ section for five (5) times to enable ‘Developer options‘ on your device.
    Install OTA Updates on Rooted Android Devices - Enable Developer Options
  5. Once enabled, go to Settings → System → Developer options.
  6. Scroll down and turn off the Automatic system updates toggle.
    Install OTA Updates on Rooted Android Devices - Disable Automatic System Updates

Step 2: Restore Stock Boot Image When an OTA is Available

Now your Android device is rooted and an OTA update notification has arrived. When an OTA update is available, you will need to restore the stock boot image (and dtbo, if applicable) and uninstall Magisk from the current slot.

Restore Stock Boot Image in Magisk Manager
Stock Boot Image Restoration Done in Magisk Manager

To do so, go to the app drawer and launch the ‘Magisk Manager‘ app. Then tap on the UNINSTALL button and select the RESTORE IMAGES option. You should now see the “Restoration done!” message on the screen.

Do not! Absolutely do not reboot your phone now, or else Magisk will be uninstalled completely.

Step 3: Install the OTA Update on your Rooted Android Device

Now, with the stock boot image restored, you can freely install the OTA update on your rooted Android device. Simply go to the device Settings → System → System update, and press the ‘Download and Install‘ button.

Install the OTA update on your Rooted Android device

Once the installation finishes, you shall be prompted to reboot your phone. DO NOT REBOOT YOUR DEVICE YET! Simply move ahead to the next step.

Step 4 [Optional]: Retain TWRP Recovery after OTA Installation

This is an optional step and only to followed if you installed Magisk using TWRP recovery (Method 1). It will allow you to retain TWRP recovery after installing the OTA update.

For this, you will need to download and install “TWRP A/B Retention Script” (by XDA Recognized Developer osm0sis) from within Magisk Manager. To do so, follow the steps below:

  1. Go to the app drawer and launch the Magisk Manager app.
  2. Tap on the menu icon (3-horizontal lines) on the top-left of the app’s window.
    Download TWRP A/B Retention Script from Magisk Manager
  3. Select ‘Downloads‘.
  4. Now search the modules repository for “TWRP A/B Retention Script“.
    Install TWRP A/B Retention Script
  5. Next, tap on the download icon and select the Install option.

The script should now be installed on your phone. There’s nothing more you’d need to do for this now. The script will do its job and preserve TWRP after the OTA installation finishes.

Step 5: Preserve Magisk Root after OTA Installation

The final step to install Magisk to the inactive slot. This will preserve root after the OTA installation is finished and the device is rebooted into the updated OS.

  1. Open Magisk Manager.
    Install Magisk to Inactive Slot - Tap on the Install button
  2. Tap on the ‘INSTALL‘ button and select the ‘INSTALL‘ option.
  3. Select ‘Install to Inactive Slot (After OTA)‘ when prompted to choose the installation method.
    Install Magisk to Inactive Slot in Magisk Manager
  4. Finally, tap on ‘YES‘ to confirm when the warning message appears on the screen.
  5. Magisk will now be installed on the inactive slot.
    Install Magisk to Inactive Slot - Reboot your device
  6. Once the process finishes, tap on the ‘Reboot‘ button.

Magisk Manager will now force-switch your Android device to the inactive slot where the OTA update was installed. This will help prevent any post-OTA verifications that the system might have carried.

That’s it! You have just successfully installed an OTA update on your Android device rooted with Magisk. So when the next time an OTA update arrives, you can follow the same instructions to upgrade your device, and that too without losing root.

If you have any questions regarding the instructions/procedure listed here, feel free to let us know. We shall try to help you to the best of our knowledge. Make sure that you mention your device’s name (model number if applicable) and the software build number installed. This will allow us to help you quickly and more efficiently.

(Source: Official Magisk documentation)



Configurations of the OTA Library.


Some configuration settings are C pre-processor constants, and some are function-like macros for logging. They can be set with a in the config file (ota_config.h) or by using a compiler option such as -D in gcc.

Define this macro to build the OTA library without the custom config file ota_config.h. Without the custom config, the OTA library builds with default values of config macros defined in ota_config_defaults.h file.

If a custom config is provided, then OTA_DO_NOT_USE_CUSTOM_CONFIG should not be defined.

The number of words allocated to the stack for the OTA agent. The configuration parameter specifies the size of the stack that will be allocated to the task being created (the size is specified in words, not bytes!). The amount of stack required is dependent on the application specific parameters, for more information Link

Possible values: Any positive 32 bit integer.
Default value: Varies by platform

The OTA agent task priority. Normally it runs at a low priority. For more information Link.

Possible values: 0 to ( configMAX_PRIORITIES – 1 )
Default value: Varies by platform.

Log base 2 of the size of the file data block message (excluding the header). Possible values: Any unsigned 32 integer.
Default value: '12'

The maximum number of data blocks requested from OTA streaming service.

This configuration parameter is sent with data requests and represents the maximum number of data blocks the service will send in response. The maximum limit for this must be calculated from the maximum data response limit (128 KB from service) divided by the block size. For example if block size is set as 1 KB then the maximum number of data blocks that we can request is 128/1 = 128 blocks. Configure this parameter to this maximum limit or lower based on how many data blocks response is expected for each data requests.

Possible values: Any unsigned 32 integer value greater than 0.
Default value: '1'

Following two diagrams compare otaconfigLOG2_FILE_BLOCK_SIZE set to 10 (1 KB) and 12 (4 KB) with otaconfigMAX_NUM_BLOCKS_REQUEST set to maximum i.e 128 and 32 respectively.

otaconfigLOG2_FILE_BLOCK_SIZE is 10 ( 1KB )
otaconfigMAX_NUM_BLOCKS_REQUEST is 128

otaconfigLOG2_FILE_BLOCK_SIZE is 12 ( 4KB )

Milliseconds to wait for the self test phase to succeed before we force reset. Possible values: Any unsigned 32 integer.
Default value: '16000'

Milliseconds to wait before requesting data blocks from the OTA service if nothing is happening.

The wait timer is reset whenever a data block is received from the OTA service so we will only send the request message after being idle for this amount of time.

Possible values: Any unsigned 32 integer.
Default value: '10000'

The maximum allowed length of the thing name used by the OTA agent.

AWS IoT requires Thing names to be unique for each device that connects to the broker. Likewise, the OTA agent requires the developer to construct and pass in the Thing name when initializing the OTA agent. The agent uses this size to allocate static storage for the Thing name used in all OTA base topics. Namely $aws/things/thingName

Possible values: Any unsigned 32 integer.
Default value: '64'

The maximum number of requests allowed to send without a response before we abort.

This configuration parameter sets the maximum number of times the requests are made over the selected communication channel before aborting and returning error.

Possible values: Any unsigned 32 integer.
Default value: '32'

The number of data buffers reserved by the OTA agent.

This configurations parameter sets the maximum number of static data buffers used by the OTA agent for job and file data blocks received.

Possible values: Any unsigned 32 integer.
Default value: '1'

How frequently the device will report its OTA progress to the cloud.

Device will update the job status with the number of blocks it has received every certain number of blocks it receives. For example, 64 means device will update job status every 64 blocks it receives.

Possible values: Any unsigned 32 integer.
Default value: '64'

Flag to enable booting into updates that have an identical or lower version than the current version.

Set this configuration parameter to '1' to disable version checks. This allows updates to an identical or lower version. This is provided for testing purpose and it's recommended to always update to higher version and keep this configuration disabled.

Possible values: Any unsigned 32 integer.
Default value: '0'

The protocol selected for OTA control operations.

This configurations parameter sets the default protocol for all the OTA control operations like requesting OTA job, updating the job status etc.
Only MQTT is supported at this time for control operations.

Possible values: OTA_CONTROL_OVER_MQTT
Default value: 'OTA_CONTROL_OVER_MQTT'

The protocol selected for OTA data operations.

This configurations parameter sets the protocols selected for the data operations like requesting file blocks from the service.

Possible values:
Enable data over MQTT - ( OTA_DATA_OVER_MQTT )
Enable data over HTTP - ( OTA_DATA_OVER_HTTP )
Enable data over both MQTT & HTTP - ( OTA_DATA_OVER_MQTT | OTA_DATA_OVER_HTTP )
Default value: 'OTA_DATA_OVER_MQTT'

The preferred protocol selected for OTA data operations.

Primary data protocol will be the protocol used for downloading file if more than one protocol is selected while creating OTA job.

Possible values:
Default value: 'OTA_DATA_OVER_MQTT'

Macro that is called in the OTA library for logging "Error" level messages. To enable error level logging in the OTA library, this macro should be mapped to the application-specific logging implementation that supports error logging.

This logging macro is called in the OTA library with parameters wrapped in double parentheses to be ISO C89/C90 standard compliant. For a reference POSIX implementation of the logging macros, refer to the ota default config file, and the logging-stack in demos folder of the AWS IoT Embedded C SDK repository.

Default value: Error logging is turned off, and no code is generated for calls to the macro in the OTA library on compilation.

Macro that is called in the OTA library for logging "Warning" level messages. To enable warning level logging in the OTA library, this macro should be mapped to the application-specific logging implementation that supports warning logging.

This logging macro is called in the OTA library with parameters wrapped in double parentheses to be ISO C89/C90 standard compliant. For a reference POSIX implementation of the logging macros, refer to the ota default config file, and the logging-stack in demos folder of the AWS IoT Embedded C SDK repository.

Default value: Warning logging is turned off, and no code is generated for calls to the macro in the OTA library on compilation.

Macro that is called in the OTA library for logging "Info" level messages. To enable info level logging in the OTA library, this macro should be mapped to the application-specific logging implementation that supports info logging.

This logging macro is called in the OTA library with parameters wrapped in double parentheses to be ISO C89/C90 standard compliant. For a reference POSIX implementation of the logging macros, refer to the ota default config file, and the logging-stack in demos folder of the AWS IoT Embedded C SDK repository.

Default value: Info logging is turned off, and no code is generated for calls to the macro in the OTA library on compilation.

Macro that is called in the OTA library for logging "Debug" level messages. To enable Debug level logging in the OTA library, this macro should be mapped to the application-specific logging implementation that supports debug logging.

This logging macro is called in the OTA library with parameters wrapped in double parentheses to be ISO C89/C90 standard compliant. For a reference POSIX implementation of the logging macros, refer to the ota default config file, and the logging-stack in demos folder of the AWS IoT Embedded C SDK repository.

Default value: Debug logging is turned off, and no code is generated for calls to the macro in the OTA library on compilation.


Based ota block

Android OTA updates

  • Android OTA UpdatesFTF 2016


    Gary Bisson

    Embedded Software Engineer


    1. Introduction

    2. Update Mechanism

    3. Update package

    4. Recovery Console

    5. OTA Application

    6. Conclusion


    Founded in 2003 Designs and manufactures ARM-based Single BoardComputers (SBC) and System on Modules (SoM)

    Targets the general embedded market Focus on customer service and fast turn-around time Reference platforms & custom designs 16,000 Square Foot Facility in Chandler, AZ

    Android OTA Updates 3 / 55

  • Introduction


    Importance of OTA updates in the field Cover all aspects

    I From update creation to download to flashing

    Past experiencesI Customers/Users feedback

    Practical approachI i.MX awareness

    I Demonstration on actual HW

    Android OTA Updates 5 / 55 Introduction


    Using HW to demonstrate the different steps Nitrogen6_MAX board Android Lollipop 5.1.1

    I Boundary Devices 5.1.1 release

    Android OTA Updates 6 / 55 Introduction


    bootI Kernel + device tree + bootargs + ramdisk

    I Raw partition created with mkbootimg

    I Actual ext4 partition for Boundary Devices

    recoveryI Same as boot with recovery ramdisk

    systemI Contains the entire OS (framework, UI, stock apps)

    Android OTA Updates 7 / 55 Introduction


    cacheI Frequently accessed data and OTA updates

    miscI Used to communicate between RC and the bootloader

    deviceI Contains assets and customizations

    dataI Users data: apps, settings, contacts

    Android OTA Updates 8 / 55 Introduction

  • Update Mechanism


    1. Download update package

    2. Check package consistency/signature

    3. Reboot into Recovery Console (RC)

    4. Veritying signature (again)

    5. Apply update

    6. Reboot into Android

    7. Run recovery update script (optional)

    Android OTA Updates 10 / 55 Update Mechanism


    Not all partitions updated by defaultI /boot and /system

    I /recovery updated at reboot if script provided

    Some partitions can be wipedI /cache and /data

    I Option available in Settings app

    Can be customizedI To flash the bootloader for instance

    Android OTA Updates 11 / 55 Update Mechanism


    Using adb sideload

    1 $ adb reboot recovery2 [Press VOL_UP and SEARCH]3 [Select 'apply update from ADB']4 $ adb sideload

    Android OTA Updates 12 / 55 Update Mechanism


    Uploading the update manually

    1 $ adb push /cache/2 $ adb shell 'mkdir /cache/recovery/'3 $ adb shell 'echo "--update_package=/cache/" > /cache/

    recovery/command'4 $ adb reboot recovery

    Android OTA Updates 13 / 55 Update Mechanism


    ReleasetoolsI Create digitally signed software updates

    I Integrated in Android build system

    Recovery Console (RC)I Alternate boot environment

    I Verify & Apply SW updates

    I Minimalistic ramdisk

    I Can perform Factory Data Reset

    I Hidden menu for manual interaction

    Android OTA Updates 14 / 55 Update Mechanism


    android.os.RecoverySystem APIsI Framework APIs to verify & install SW updates

    I Writes RC command files into /cache/recovery andreboots into Recovery Console

    I Also used to engage Factory Data Reset

    UpdaterI SW update logic, binary inside SW update package

    I AOSP implementation runs script in Edify language

    I Platform-specific tasks implemented in plug-ins

    SW Update UI Intent from Settings applicationI Providing a flexible approach

    Android OTA Updates 15 / 55 Update Mechanism


    Communication between Android and bootloaderI Vendor-specific

    Client-side OTA applicationI Mechanism to check and download updates

    I Notification UI that SW Updates are available

    I Assumed specific to each use case

    Remote server backend to host updatesI Assumed specific to each infrastructure

    Android OTA Updates 16 / 55 Update Mechanism


    NXP is providing everything you need to get started!

    Communication between Android and bootloaderI Using the Secure Non-Volatile Storage (SNVS)

    I See kernel_imx/arch/arm/mach-imx/system.c

    Client-side OTA applicationI FSLOta application

    I Just fetching updates when explicitly started

    I Good starting point

    I See packages/apps/fsl_imx_demo/FSLOta/

    Remote server backend to host updatesI FSLOta just requires a standard HTTP server

    Android OTA Updates 17 / 55 Update Mechanism


    No support for disk re-partitioningI Not impossible but not recommended

    One update applied at a time Device cant be used while updates are applied

    Android OTA Updates 18 / 55 Update Mechanism

  • Update package


    A full update is a zip archive containing the entire final state of

    the device (system, boot, and recovery partitions).

    Automatic creation:1 $ . build/ && lunch nitrogen6x-eng2 $ make dist3 $ ls $OUT/

    Manual creation from target files:1 $ ./build/tools/releasetools/ota_from_target_files \2 out/dist/

    Android OTA Updates 20 / 55 Update package


    An incremental update is a zip archive containing a set of

    binary patches to be applied to the data already on the device.

    This can result in considerably smaller update packages.

    Requires to keep the previous update target files:1 $ ./build/tools/releasetools/ota_from_target_files \2 -i \3

    Android OTA Updates 21 / 55 Update package








    Android OTA Updates 22 / 55 Update package


    file_contextsI Contains the SELinux labels to apply after update

    META-INFI Hosts the package manifest file and code signatures

    I Like META-INF of an APK file + few files

    I com/android/otacert is the update signing certificate

    I com/google/android/ contains important binaries updater-script


    Android OTA Updates 23 / 55 Update package


    Created by Written in edify scripting language Customization with releasetools extensions


    I FullOTA_InstallBegin() Called at the beginning of full OTA installation

    I FullOTA_InstallEnd() Called at the end of full OTA installation

    I See all options in

    Android OTA Updates 24 / 55 Update package


    Scripting language to specify SW update tasks All functions implemented in C

    I See bootable/recovery/updater/install.c:mount/unmount/show_progress/delete/symlink/getprop/wipe_cache/ui_print...

    Cannot declare functions in an Edify scriptI Additional functions implemented in plug-ins

    See bootable/recovery/edify/README

    Android OTA Updates 25 / 55 Update package


    AOSP updater implementation:I See bootable/recovery/updater/updater.c

    Forked from the recovery process Loads the updater-script and parses the commands Platform-specific updater capabilities implemented inplug-ins

    Not strictly required to use the updater implementation

    Android OTA Updates 26 / 55 Update package


    Multiple updater plug-ins can be definedI Add TARGET_RECOVERY_UPDATER_LIBS

    I Each library needs a registration function

    void Register_$LOCAL_MODULE()

    Calls RegisterFunction() for each command

    Use bootable/recovery/updater/install.c as a guide Example: add a function to flash the bootloader

    Android OTA Updates 27 / 55 Update package

  • UPDATER PLUG-INS example:1 include $(CLEAR_VARS)2 LOCAL_SRC_FILES := recovery_updater.c3 LOCAL_C_INCLUDES += bootable/recovery4 LOCAL_MODULE := librecovery_updater_imx5 include $(BUILD_STATIC_LIBRARY) addition:1 TARGET_RECOVERY_UPDATER_LIBS += librecovery_updater_imx

    Android OTA Updates 28 / 55 Update package


    plug-in snippet:1 #include "edify/expr.h"2

    3 Value* CustomImxFn(const char* name, State* state,4 int argc, Expr* argv[]) {5 if (argc != 2) {6 return ErrorAbort(state, "%s expects 2 args", name);7 }8 // Do custom work9 }


    11 void Register_librecovery_updater_imx() {

  • Sours:
    Jc La Nevula - Buscate Otro Ft Atomic Otro Way (VIDEO OFICIAL)

    Android OTA Update

    Android OTA Update

    • ref

    • Devices have a special recovery partition with the software needed to unpack a downloaded update package and apply it to the rest of the system.

    • OTA updates are designed to upgrade the underlying operating system and the read-only apps installed on the system partition; these updates do not affect applications installed by the user from Google Play.
    • The flash space on an Android device typically contains the following partitions.
    • ,Contains the Linux kernel and a minimal root filesystem (loaded into a RAM disk). It mounts system and other partitions and starts the runtime located on the system partition.
    • ,Contains system applications and libraries that have source code available on Android Open Source Project (AOSP). During normal operation, this partition is mounted read-only; its contents change only during an OTA update.
    • ,Contains system applications and libraries that do not have source code available on Android Open Source Project (AOSP). During normal operation, this partition is mounted read-only; its contents change only during an OTA update.
    • ,Stores the data saved by applications installed by the user, etc. This partition is not normally touched by the OTA update process.
    • ,Temporary holding area used by a few applications (accessing this partition requires special app permissions) and for storage of downloaded OTA update packages. Other programs use this space with the expectation that files can disappear at any time. Some OTA package installations may result in this partition being wiped completely.
    • ,Contains a second complete Linux system, including a kernel and the special recovery binary that reads a package and uses its contents to update the other partitions.
    • ,Tiny partition used by recovery to stash some information away about what it's doing in case the device is restarted while the OTA package is being applied.

    • Life of an OTA update

    • Device performs regular check in with OTA servers and is notified of the availability of an update, including the URL of the update package and a description string to show the user.
    • Update downloads to a cache or data partition, and its cryptographic signature is verified against the certificates in . User is prompted to install the update.
    • Device reboots into recovery mode, in which the kernel and system in the recovery partition are booted instead of the kernel in the boot partition.
    • Recovery binary is started by . It finds command-line arguments in that point it to the downloaded package.
    • Recovery verifies the cryptographic signature of the package against the public keys in (part of the RAM disk contained in the recovery partition).
    • Data is pulled from the package and used to update the , , and/or partitions as necessary. One of the new files left on the partition contains the contents of the new recovery partition.
    • Device reboots normally.(a)The newly updated boot partition is loaded, and it mounts and starts executing binaries in the newly updated system partition.(b)As part of normal startup, the system checks the contents of the recovery partition against the desired contents (which were previously stored as a file in /system). They are different, so the recovery partition is reflashed with the desired contents. (On subsequent boots, the recovery partition already contains the new contents, so no reflash is necessary.)

    • The tool provided in can build two types of package: and . The tool takes the file produced by the Android build system as input.

    • A full update is one where the entire final state of the device (system, boot, and recovery partitions) is contained in the package. As long as the device is capable of receiving the package and booting the recovery system, the package can install the desired build regardless of the current state of the device.
    # first, build the target-files .zip % . build/ && lunch tardis-eng % mkdir dist_output % make dist DIST_DIR=dist_output [...] % ls -l dist_output/*target_files* -rw-r----- 1 user eng 69965275 Sep 29 15:51 # The target-files .zip contains everything needed to construct OTA packages. % ./build/tools/releasetools/ota_from_target_files \ dist_output/ unzipping target target-files... done. % ls -l -rw-r----- 1 user eng 62236561 Sep 29 15:58
    • An incremental update contains a set of binary patches to be applied to the data already on the device. This can result in considerably smaller update packages:(a)Files that have not changed don't need to be included.(b)Files that have changed are often very similar to their previous versions, so the package need only contain an encoding of the differences between the two files.You can install the incremental update package only on a device that has the old or source build used when constructing the package. To build an incremental update, you need the target_files .zip from the previous build (the one you want to update from) as well as the target_files .zip from the new build.
    % ./build/tools/releasetools/ota_from_target_files \ -i \ # make incremental from this older version dist_output/ unzipping target target-files... unzipping source target-files... [...]done. % ls -l -rw-r----- 1 user eng 1175314 Sep 29 16:10 #To generate a block-based OTA for subsequent updates, pass the --block option to ota_from_target_files.
    • An update package (, ) is a file that contains the executable binary . After verifying the signature on the package, recovery extracts this binary to and runs it, passing the following arguments:
    • Update binary API version number. If the arguments passed to the update binary change, this number is incremented.
    • File descriptor of the command pipe. The update program can use this pipe to send commands back to the recovery binary (mostly for UI changes such as indicating progress to the user).
    • Filename of the update package .zip file. A recovery package can use any statically-linked binary as the update binary. The OTA package construction tools use the updater program (source in bootable/recovery/updater), which provides a simple scripting language that can do many installation tasks. You can substitute any other binary running on the device.

    • Android 5.0 and later versions use updates to ensure that each device uses the exact same partition. Instead of comparing individual files and computing binary patches, handles the entire partition as one file and computes a single binary patch, ensuring the resultant partition contains exactly the intended bits. This allows the device system image to achieve the same state via or OTA.Android 4.4 and earlier versions used updates, which ensured devices contained similar , but allowed metadata such as timestamps and the layout of the underlying storage to vary between devices based on the update method.

    • Because ensures that each device uses the same partition, it enables the use of to cryptographically sign the system partition.

    • During a file-based OTA, Android attempts to change the contents of the system partition at the filesystem layer (on a file-by-file basis). The update is not guaranteed to write files in a consistent order, have a consistent last modified time or superblock, or even place the blocks in the same location on the block device. For this reason, file-based OTAs fail on a dm-verity-enabled device; after the OTA attempt, the device does not boot.During a block-based OTA, Android serves the device the difference between the two block images (rather than two sets of files). The update checks a device build against the corresponding build server at the block level (below the filesystem) using one of the following methods:

    • . Copying the full system image is simple and makes patch generation easy but also generates large images that can make applying patches expensive.
    • . Using a binary differ tool generates smaller images and makes patch application easy, but is memory-intensive when generating the patch itself.

    places the exact same bits on the device as a , so flashing is compatible with .

    • In general, updates are larger than updates due to:
    • Data preservation. Block-based OTAs preserve more data (, , , etc.) than file-based OTA.
    • Computation algorithm differences. In a , if a file path is identical in both builds, the OTA package contains no data for that file. In a , determining little or no change in a file depends on the quality of the patch computation algorithm and layout of file data in both source and target system.

    If a file is corrupted, a file OTA succeeds as long as it doesn't touch the corrupted file, but a block OTA fails if it detects any corruption on the system partition.

    The system builds the updater binary from and uses it in an OTA package.The package itself is a file (, ) that contains the executable binary .Updater contains several builtin functions and an interpreter for an extensible scripting language (edify) that supports commands for typical update-related tasks. Updater looks in the package .zip file for a script in the file .Using the edify script and/or builtin functions is not a common activity, but can be helpful if you need to debug the update file.

    An edify script is a single expression in which all values are strings. Empty strings are false in a Boolean context and all other strings are true. Edify supports the following operators (with the usual meanings):

    (expr ) expr + expr # string concatenation, not integer additionexpr== expr expr != expr expr && expr expr || expr ! expr if expr then expr endif if expr then expr else expr endif function_name(expr, expr,...) expr; expr
    • The recovery system includes several hooks for inserting device-specific code so that OTA updates can also update parts of the device other than the Android system (e.g., the baseband or radio processor).The partition map file is specified by ; this file is used by both the recovery binary and the package-building tools. You can specify the name of the map file in in .

    A sample partition map file might look like this:

    device/yoyodyne/tardis/recovery.fstab # mount point fstype device [device2] [options (3.0+ only)] /sdcard vfat /dev/block/mmcblk0p1 /dev/block/mmcblk0 /cache yaffs2 cache /misc mtd misc /boot mtd boot /recovery emmc /dev/block/platform/s3c-sdhci.0/by-name/recovery /system ext4 /dev/block/platform/s3c-sdhci.0/by-name/system length=-4096 /data ext4 /dev/block/platform/s3c-sdhci.0/by-name/userdata

    There are five supported filesystem types:,,,,.

    Starting in Android 3.0, the file gains an additional optional field, options. Currently the only defined option is length , which lets you explicitly specify the length of the partition. This length is used when reformatting the partition (e.g., for the userdata partition during a data wipe/factory reset operation, or for the system partition during installation of a full OTA package). If the length value is negative, then the size to format is taken by adding the length value to the true partition size. For instance, setting "length=-16384" means the last 16k of that partition will not be overwritten when that partition is reformatted. This supports features such as encryption of the userdata partition (where encryption metadata is stored at the end of the partition that should not be overwritten).

    • Android OS images use cryptographic signatures in two places:

    • Each file inside the image must be signed. Android's Package Manager uses an signature in two ways:

      • When an application is replaced, it must be signed by the same key as the old application in order to get access to the old application's data. This holds true both for updating user apps by overwriting the , and for overriding a system app with a newer version installed under .
      • If two or more applications want to share a user ID (so they can share data, etc.), they must be signed with the same key.
    • OTA update packages must be signed with one of the keys expected by the system or the installation process will reject them.

    • To generate your own unique set of release-keys, run these commands from the root of your Android tree:

    #$subject should be changed to reflect your organization's information.subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/[email protected]' mkdir ~/.android-certs for x in releasekey platform shared media; do\ ./development/tools/make_key ~/.android-certs/$x"$subject"; \done
    • To generate a release image, use:
    make dist ./build/tools/releasetools/sign_target_files_apks \ -o \ # explained in the next section -d ~/.android-certs out/dist/*-target_files-*.zip \

    The script takes a as input and produces a new in which all the .apks have been signed with new keys. The newly signed images can be found under in .

    • A signed target-files zip can be converted into a signed OTA update zip using the following procedure:
    ./build/tools/releasetools/ota_from_target_files \ -k ~/.android-certs/releasekey \ \
    • Update packages received from the main system are typically verified twice: once by the main system, using the method in the android API, and then again by recovery. The RecoverySystem API checks the signature against public keys stored in the main system, in the file (by default). Recovery checks the signature against public keys stored in the recovery partition RAM disk, in the file .

    • Each key comes in two files: the certificate, which has the extension .x509.pem, and the private key, which has the extension .pk8. The private key should be kept secret and is needed to sign a package. The key may itself be protected by a password. The certificate, in contrast, contains only the public half of the key, so it can be distributed widely. It is used to verify a package has been signed by the corresponding private key. The standard Android build uses four keys, all of which reside in build/target/product/security:

    • :Generic default key for packages that do not otherwise specify a key.
    • :Test key for packages that are part of the core platform.
    • :Test key for things that are shared in the home/contacts process.
    • :Test key for packages that are part of the media/download system. Individual packages specify one of these keys by setting in their file. (testkey is used if this variable is not set.) You can also specify an entirely different key by pathname, e.g.:
    #device/yoyodyne/apps/SpecialApp/[...] LOCAL_CERTIFICATE := device/yoyodyne/security/special #Now the build uses the device/yoyodyne/security/special.{x509.pem,pk8} key to sign SpecialApp.apk. The build can use only private keys that are not password protected.
    • Android uses 2048-bit RSA keys with public exponent 3. You can generate certificate/private key pairs using the openssl tool from
    # generate RSA key % openssl genrsa -3 -out temp.pem 2048 Generating RSA private key, 2048 bit long modulus ....+++ .....................+++ e is 3 (0x3)# create a certificate with the public part of the key % openssl req -new -x509 -key temp.pem -out releasekey.x509.pem \ -days 10000 \ -subj '/C=US/ST=California/L=San Narciso/O=Yoyodyne, Inc./OU=Yoyodyne Mobility/CN=Yoyodyne/[email protected]'# create a PKCS#8-formatted version of the private key % openssl pkcs8 -in temp.pem -topk8 -outform DER -out releasekey.pk8 -nocrypt # securely delete the temp.pem file % shred --remove temp.pem

    The openssl pkcs8 command given above creates a file with no password, suitable for use with the build system. To create a secured with a password (which you should do for all actual release keys), replace the argument with ; then openssl will encrypt the private key with a password read from standard input.

    • Once you have, you need to create the image so you can put it onto a device. To create the signed image from the target files, run the following command from the root of the Android tree:

    The resulting file, , contains all the files. To load an image onto a device, use fastboot as follows:.


    You will also like:


    327 328 329 330 331