fdroidserver.common module¶

class fdroidserver.common.ClonedZipInfo(zinfo)¶

Bases: ZipInfo

Hack to allow fully cloning ZipInfo instances.

The zipfile library has some bugs that prevent it from fully cloning ZipInfo entries. https://bugs.python.org/issue43547

Attributes:

CRC
comment
compress_size
compress_type
create_system
create_version
date_time
external_attr
extra
extract_version
file_size
filename
flag_bits
header_offset
internal_attr
orig_filename
reserved
volume

Methods

`FileHeader`([zip64])	Return the per-file header as a bytes object.
`from_file`(filename[, arcname, strict_timestamps])	Construct an appropriate ZipInfo for a file on the filesystem.
`is_dir`()	Return True if this archive member is a directory.

CRC¶

comment¶

compress_size¶

compress_type¶

create_system¶

create_version¶

date_time¶

external_attr¶

extra¶

extract_version¶

file_size¶

filename¶

flag_bits¶

header_offset¶

internal_attr¶

orig_filename¶

reserved¶

volume¶

class fdroidserver.common.Encoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)¶

Bases: JSONEncoder

Methods

`default`(obj)	Implement this method in a subclass such that it returns a serializable object for `o`, or calls the base implementation (to raise a `TypeError`).
`encode`(o)	Return a JSON string representation of a Python data structure.
`iterencode`(o[, _one_shot])	Encode the given object and yield each string representation as available.

default(obj)¶

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

fdroidserver.common.FDroidPopen(commands, cwd=None, envs=None, output=True, stderr_to_stdout=True)¶

Run a command and capture the possibly huge output as a str.

Parameters:

commands: command and argument list like in subprocess.Popen
cwd: optionally specifies a working directory
envs: a optional dictionary of environment variables and their values

Returns:

A PopenResult.

fdroidserver.common.FDroidPopenBytes(commands, cwd=None, envs=None, output=True, stderr_to_stdout=True)¶

Run a command and capture the possibly huge output as bytes.

Parameters:

commands: command and argument list like in subprocess.Popen
cwd: optionally specifies a working directory
envs: a optional dictionary of environment variables and their values

Returns:

A PopenResult.

class fdroidserver.common.KnownApks¶

Bases: object

Permanent store of existing APKs with the date they were added.

This is currently the only way to permanently store the “updated” date of APKs.

Methods

`getapp`(apkname)	Look up information - given the 'apkname'.
`getlatest`(num)	Get the most recent 'num' apps added to the repo, as a list of package ids with the most recent first.
`recordapk`(apkName, app[, default_date])	Record an APK (if it's new, otherwise does nothing).

writeifchanged

getapp(apkname)¶

Look up information - given the ‘apkname’.

Returns (app id, date added/None). Or returns None for an unknown apk.

getlatest(num)¶: Get the most recent ‘num’ apps added to the repo, as a list of package ids with the most recent first.

recordapk(apkName, app, default_date=None)¶

Record an APK (if it’s new, otherwise does nothing).

Returns:

datetime: the date it was added as a datetime instance.

writeifchanged()¶

class fdroidserver.common.PopenResult(returncode=None, output=None)¶: Bases: object

fdroidserver.common.SdkToolsPopen(commands, cwd=None, output=True)¶

fdroidserver.common.ant_subprojects(root_dir)¶

fdroidserver.common.apk_extract_signatures(apkpath, outdir)¶

Extract a signature files from APK and puts them into target directory.

Parameters:

apkpath: location of the apk
outdir: older where the extracted signature files will be stored

References

fdroidserver.common.apk_has_v1_signatures(apkfile)¶: Test whether an APK has v1 signature files.

fdroidserver.common.apk_implant_signatures(apkpath, outpath, manifest)¶

Implant a signature from metadata into an APK.

Note: this changes there supplied APK in place. So copy it if you need the original to be preserved.

Parameters:

apkpath: location of the unsigned apk
outpath: location of the output apk

References

fdroidserver.common.apk_parse_release_filename(apkname)¶

Parse the name of an APK file according the F-Droids APK naming scheme.

WARNING: Returned values don’t necessarily represent the APKs actual properties, the are just paresed from the file name.

Returns:

Tuple: A triplet containing (appid, versionCode, signer), where appid should be the package name, versionCode should be the integer represion of the APKs version and signer should be the first 7 hex digists of the sha256 signing key fingerprint which was used to sign this APK.

fdroidserver.common.apk_signer_fingerprint(apk_path)¶

Obtain sha256 signing-key fingerprint for APK.

Extracts hexadecimal sha256 signing-key fingerprint string for a given APK.

Parameters:

apk_path: path to APK

Returns:

signature fingerprint

fdroidserver.common.apk_signer_fingerprint_short(apk_path)¶

Obtain shortened sha256 signing-key fingerprint for APK.

Extracts the first 7 hexadecimal digits of sha256 signing-key fingerprint for a given pkcs7 APK.

Parameters:

apk_path: path to APK

Returns:

shortened signing-key fingerprint

fdroidserver.common.apk_strip_v1_signatures(signed_apk, strip_manifest=False)¶

Remove signatures from APK.

Parameters:

signed_apk: path to APK file.
strip_manifest: when set to True also the manifest file will be removed from the APK.

fdroidserver.common.app_matches_packagename(app, package)¶

fdroidserver.common.assert_config_keystore(config)¶: Check weather keystore is configured correctly and raise exception if not.

fdroidserver.common.auto_install_ndk(build)¶

Auto-install the NDK in the build, this assumes its in a buildserver guest VM.

Download, verify, and install the NDK version as specified via the “ndk:” field in the build entry. As it uncompresses the zipball, this forces the permissions to work for all users, since this might uncompress as root and then be used from a different user.

This needs to be able to install multiple versions of the NDK, since this is also used in CI builds, where multiple fdroid build –onserver calls can run in a single session. The production buildserver is reset between every build.

The default ANDROID_SDK_ROOT base dir of /opt/android-sdk is hard-coded in buildserver/Vagrantfile. The $ANDROID_HOME/ndk subdir is where Android Studio will install the NDK into versioned subdirs. https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41

Also, r10e and older cannot be handled via this mechanism because they are packaged differently.

fdroidserver.common.calculate_IPFS_cid(filename)¶

Calculate the IPFS CID of a file and add it to the index.

uses ipfs_cid package at https://packages.debian.org/sid/ipfs-cid Returns CIDv1 of a file as per IPFS recommendation

fdroidserver.common.calculate_math_string(expr)¶

fdroidserver.common.check_system_clock(dt_obj, path)¶

Check if system clock is updated based on provided date.

If an APK has files newer than the system time, suggest updating the system clock. This is useful for offline systems, used for signing, which do not have another source of clock sync info. It has to be more than 24 hours newer because ZIP/APK files do not store timezone info

fdroidserver.common.compare_apks(apk1, apk2, tmp_dir, log_dir=None)¶

Compare two apks.

Returns:

None if the APK content is the same (apart from the signing key),
otherwise a string describing what’s different, or what went wrong when
trying to do the comparison.

fdroidserver.common.config_type_check(path, data)¶

fdroidserver.common.deploy_build_log_with_rsync(appid, vercode, log_content)¶

Upload build log of one individual app build to an fdroid repository.

Parameters:

appid: package name for dientifying to which app this log belongs.
vercode: version of the app to which this build belongs.
log_content: Content of the log which is about to be posted. Should be either a string or bytes. (bytes will be decoded as ‘utf-8’)

fdroidserver.common.ensure_final_value(packageName, arsc, value)¶

Ensure incoming value is always the value, not the resid.

androguard will sometimes return the Android “resId” aka Resource ID instead of the actual value. This checks whether the value is actually a resId, then performs the Android Resource lookup as needed.

fdroidserver.common.fetch_real_name(app_dir, flavours)¶: Retrieve the package name. Returns the name, or None if not found.

fdroidserver.common.file_entry(filename, hash_value=None)¶

fdroidserver.common.fill_config_defaults(thisconfig)¶

Fill in the global config dict with relevant defaults.

For config values that have a path that can be expanded, e.g. an env var or a ~/, this will store the original value using “_orig” appended to the key name so that if the config gets written out, it will preserve the original, unexpanded string.

fdroidserver.common.find_apksigner(config)¶

Search for the best version apksigner and adds it to the config.

Returns the best version of apksigner following this algorithm:

use config[‘apksigner’] if set
try to find apksigner in path
find apksigner in build-tools starting from newest installed going down to MINIMUM_APKSIGNER_BUILD_TOOLS_VERSION

Returns:

str: path to apksigner or None if no version is found

fdroidserver.common.find_command(command)¶: Find the full path of a command, or None if it can’t be found in the PATH.

fdroidserver.common.find_sdk_tools_cmd(cmd)¶: Find a working path to a tool from the Android SDK.

fdroidserver.common.force_exit(exitvalue=0)¶

Force exit when thread operations could block the exit.

The build command has to use some threading stuff to handle the timeout and locks. This seems to prevent the command from exiting, unless this hack is used.

fdroidserver.common.fsearch_g(/, string, pos=0, endpos=sys.maxsize)¶

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

fdroidserver.common.genkeystore(localconfig)¶

Generate a new key with password provided in localconfig and add it to new keystore.

Parameters:

localconfig

Returns:

hexed public key, public key fingerprint

fdroidserver.common.genpassword()¶: Generate a random password for when generating keys.

fdroidserver.common.get_all_gradle_and_manifests(build_dir)¶

fdroidserver.common.get_android_tools_version_log()¶: Get a list of the versions of all installed Android SDK/NDK components.

fdroidserver.common.get_android_tools_versions()¶: Get a list of the versions of all installed Android SDK/NDK components.

fdroidserver.common.get_apk_id(apkfile)¶

Extract identification information from APK.

Androguard is preferred since it is more reliable and a lot faster. Occasionally, when androguard can’t get the info from the APK, aapt still can. So aapt is also used as the final fallback method.

Parameters:

apkfile: path to an APK file.

Returns:

appid
version code
version name

fdroidserver.common.get_apk_id_aapt(apkfile)¶: Read (appid, versionCode, versionName) from an APK.

fdroidserver.common.get_apk_id_androguard(apkfile)¶

Read (appid, versionCode, versionName) from an APK.

This first tries to do quick binary XML parsing to just get the values that are needed. It will fallback to full androguard parsing, which is slow, if it can’t find the versionName value or versionName is set to a Android String Resource (e.g. an integer hex value that starts with @).

fdroidserver.common.get_apksigner_smartcardoptions(smartcardoptions)¶

fdroidserver.common.get_app_display_name(app)¶

Get a human readable name for the app for logging and sorting.

When trying to find a localized name, this first tries en-US since that his the historical language used for sorting.

fdroidserver.common.get_build_dir(app)¶: Get the dir that this app will be built in.

fdroidserver.common.get_cert_fingerprint(pubkey)¶: Generate a certificate fingerprint the same way keytool does it (but with slightly different formatting).

fdroidserver.common.get_certificate(signature_block_file)¶

Extract a DER certificate from JAR Signature’s “Signature Block File”.

Parameters:

signature_block_file: file bytes (as string) representing the certificate, as read directly out of the APK/ZIP

Returns:

A binary representation of the certificate’s public key,
or None in case of error

fdroidserver.common.get_config(opts=None)¶: Get config instace. This function takes care of initializing config data before returning it.

fdroidserver.common.get_dir_size(path_or_str)¶: Get the total size of all files in the given directory.

fdroidserver.common.get_examples_dir()¶: Return the dir where the fdroidserver example files are available.

fdroidserver.common.get_extension(filename)¶: Get name and extension of filename, with extension always lower case.

fdroidserver.common.get_file_extension(filename)¶: Get the normalized file extension, can be blank string but never None.

fdroidserver.common.get_first_signer_certificate(apkpath)¶: Get the first signing certificate from the APK, DER-encoded.

fdroidserver.common.get_gradle_subdir(build_dir, paths)¶: Get the subdir where the gradle build is based.

fdroidserver.common.get_head_commit_id(git_repo)¶: Get git commit ID for HEAD as a str.

fdroidserver.common.get_library_references(root_dir)¶

fdroidserver.common.get_local_metadata_files()¶

Get any metadata files local to an app’s source repo.

This tries to ignore anything that does not count as app metdata, including emacs cruft ending in ~

fdroidserver.common.get_metadata_files(vercodes)¶

Build a list of metadata files and raise an exception for invalid appids.

Parameters:

vercodes: version codes as returned by read_pkg_args()

Returns:

List: a list of corresponding metadata/*.yml files

fdroidserver.common.get_min_sdk_version(apk)¶

Wrap the androguard function to always return and int.

Fall back to 1 if we can’t get a valid minsdk version.

Parameters:

apk: androguard APK object

Returns:

minsdk: int

fdroidserver.common.get_native_code(apkfile)¶

Aapt checks if there are architecture folders under the lib/ folder.

We are simulating the same behaviour.

fdroidserver.common.get_ndk_version(ndk_path)¶

Get the version info from the metadata in the NDK package.

Since r11, the info is nice and easy to find in sources.properties. Before, there was a kludgey format in RELEASE.txt. This is only needed for r10e.

fdroidserver.common.get_per_app_repos()¶: Per-app repos are dirs named with the packageName of a single app.

fdroidserver.common.get_release_filename(app, build, extension=None)¶

fdroidserver.common.get_toolsversion_logname(app, build)¶

fdroidserver.common.getpaths(build_dir, globpaths)¶: Extend via globbing the paths from a field and return them as a set.

fdroidserver.common.getpaths_map(build_dir, globpaths)¶: Extend via globbing the paths from a field and return them as a map from original path to resulting paths.

fdroidserver.common.getsrclib(spec, srclib_dir, basepath=False, raw=False, prepare=True, preponly=False, refresh=True, build=None)¶

Get the specified source library.

Return the path to it. Normally this is the path to be used when referencing it, which may be a subdirectory of the actual project. If you want the base directory of the project, pass ‘basepath=True’.

spec and srclib_dir are both strings, not pathlib.Path.

fdroidserver.common.getsrclibvcs(name)¶

fdroidserver.common.getsrcname(app, build)¶

fdroidserver.common.getvcs(vcstype, remote, local)¶

Return a vcs instance based on the arguments.

remote and local can be either a string or a pathlib.Path

fdroidserver.common.is_apk_and_debuggable(apkfile)¶

Return True if the given file is an APK and is debuggable.

Parse only <application android:debuggable=””> from the APK.

Parameters:

apkfile: full path to the APK to check

fdroidserver.common.is_repo_file(filename, for_gpg_signing=False)¶: Whether the file in a repo is a build product to be delivered to users.

fdroidserver.common.is_strict_application_id(name)¶

Check whether name is a valid Android Application ID.

The Android ApplicationID is basically a Java Package Name, but with more restrictive naming rules:

It must have at least two segments (one or more dots).
Each segment must start with a letter.
All characters must be alphanumeric or an underscore [a-zA-Z0-9_].

References

https://developer.android.com/studio/build/application-id

fdroidserver.common.is_valid_package_name(name)¶

Check whether name is a valid fdroid package name.

APKs and manually defined package names must use a valid Java Package Name. Automatically generated package names for non-APK files use the SHA-256 sum.

fdroidserver.common.load_localized_config(name, repodir)¶

Load localized config files and put them into internal dict format.

This will maintain the order as came from the data files, e.g YAML. The locale comes from unsorted paths on the filesystem, so that is separately sorted.

fdroidserver.common.load_stats_fdroid_signing_key_fingerprints()¶

Load signing-key fingerprints stored in file generated by fdroid publish.

Returns:

dict: containing the signing-key fingerprints.

fdroidserver.common.local_rsync(options, fromdir, todir)¶

Rsync method for local to local copying of things.

This is an rsync wrapper with all the settings for safe use within the various fdroidserver use cases. This uses stricter rsync checking on all files since people using offline mode are already prioritizing security above ease and speed.

fdroidserver.common.manifest_paths(app_dir, flavours)¶: Return list of existing files that will be used to find the highest vercode.

fdroidserver.common.metadata_find_developer_signature(appid, vercode=None)¶

Try to find the developer signature for given appid.

This picks the first signature file found in metadata an returns its signature.

Returns:

sha256 signing key fingerprint of the developer signing key.
None in case no signature can not be found.

fdroidserver.common.metadata_find_developer_signing_files(appid, vercode)¶

Get developer signature files for specified app from metadata.

Returns:

List: of 4-tuples for each signing key with following paths: (signature_file, signature_block_file, manifest, v2_files), where v2_files is either a (apk_signing_block_offset_file, apk_signing_block_file) pair or None

fdroidserver.common.metadata_find_signing_files(appid, vercode)¶

Get a list of signed manifests and signatures.

Parameters:

appid: app id string
vercode: app version code

Returns:

List: of 4-tuples for each signing key with following paths: (signature_file, signature_block_file, manifest, v2_files), where v2_files is either a (apk_signing_block_offset_file, apk_signing_block_file) pair or None

References

fdroidserver.common.metadata_get_sigdir(appid, vercode=None)¶: Get signature directory for app.

fdroidserver.common.natural_key(s)¶

fdroidserver.common.parse_androidmanifests(paths, app)¶

Extract some information from the AndroidManifest.xml at the given path.

Returns (version, vercode, package), any or all of which might be None. All values returned are strings.

Android Studio recommends “you use UTF-8 encoding whenever possible”, so this code assumes the files use UTF-8. https://sites.google.com/a/android.com/tools/knownissues/encoding

fdroidserver.common.parse_human_readable_size(size)¶

fdroidserver.common.parse_srclib_spec(spec)¶

fdroidserver.common.parse_xml(path)¶

fdroidserver.common.place_srclib(root_dir, number, libpath)¶

fdroidserver.common.prepare_source(vcs, app, build, build_dir, srclib_dir, extlib_dir, onserver=False, refresh=True)¶

Prepare the source code for a particular build.

Parameters:

vcs: the appropriate vcs object for the application
app: the application details from the metadata
build: the build details from the metadata
build_dir: the path to the build directory, usually ‘build/app.id’
srclib_dir: the path to the source libraries directory, usually ‘build/srclib’
extlib_dir: the path to the external libraries directory, usually ‘build/extlib’

Returns:

root: is the root directory, which may be the same as ‘build_dir’ or may be a subdirectory of it.
srclibpaths: is information on the srclibs being used

fdroidserver.common.psearch_g(/, string, pos=0, endpos=sys.maxsize)¶

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

fdroidserver.common.publishednameinfo(filename)¶

fdroidserver.common.read_app_args(appid_versionCode_pairs, allapps, allow_vercodes=False)¶

Build a list of App instances for processing.

On top of what read_pkg_args does, this returns the whole app metadata, but limiting the builds list to the builds matching the appid_versionCode_pairs and vercodes specified. If no appid_versionCode_pairs are specified, then all App and Build instances are returned.

fdroidserver.common.read_config(opts=None)¶

Read the repository config.

The config is read from config_file, which is in the current directory when any of the repo management commands are used. If there is a local metadata file in the git repo, then the config is not required, just use defaults.

config.yml is the preferred form because no code is executed when reading it. config.py is deprecated and supported for backwards compatibility.

config.yml requires ASCII or UTF-8 encoding because this code does not auto-detect the file’s encoding. That is left up to the YAML library. YAML allows ASCII, UTF-8, UTF-16, and UTF-32 encodings. Since it is a good idea to manage config.yml (WITHOUT PASSWORDS!) in git, it makes sense to use a globally standard encoding.

fdroidserver.common.read_pkg_args(appid_versionCode_pairs, allow_vercodes=False)¶

No summary.

Parameters:

appids: arguments in the form of multiple appid:[vc] strings

Returns:

a dictionary with the set of vercodes specified for each package

fdroidserver.common.regsub_file(pattern, repl, path)¶

fdroidserver.common.remove_debuggable_flags(root_dir)¶

fdroidserver.common.remove_signing_keys(build_dir)¶

fdroidserver.common.replace_build_vars(cmd, build)¶

fdroidserver.common.replace_config_vars(cmd, build)¶

fdroidserver.common.retrieve_string(app_dir, string, xmlfiles=None)¶

fdroidserver.common.retrieve_string_singleline(app_dir, string, xmlfiles=None)¶

fdroidserver.common.rsync_status_file_to_repo(path, repo_subdir=None)¶: Copy a build log or status JSON to the repo using rsync.

fdroidserver.common.run_yamllint(path, indent=0)¶

fdroidserver.common.set_FDroidPopen_env(build=None)¶

Set up the environment variables for the build environment.

There is only a weak standard, the variables used by gradle, so also set up the most commonly used environment variables for SDK and NDK. Also, if there is no locale set, this will set the locale (e.g. LANG) to en_US.UTF-8.

fdroidserver.common.set_command_in_config(command)¶

Try to find specified command in the path, if it hasn’t been manually set in config.yml.

If found, it is added to the config dict. The return value says whether the command is available.

fdroidserver.common.setup_global_opts(parser)¶

fdroidserver.common.setup_status_output(start_timestamp)¶: Create the common output dictionary for public status updates.

fdroidserver.common.setup_vcs(app)¶: Checkout code from VCS and return instance of vcs and the build dir.

fdroidserver.common.sha256base64(filename)¶: Calculate the sha256 of the given file as URL-safe base64.

fdroidserver.common.sha256sum(filename)¶: Calculate the sha256 of the given file.

fdroidserver.common.sign_apk(unsigned_path, signed_path, keyalias)¶

Sign an unsigned APK, then save to a new file, deleting the unsigned.

NONE is a Java keyword used to configure smartcards as the keystore. Otherwise, the keystore is a local file. https://docs.oracle.com/javase/7/docs/technotes/guides/security/p11guide.html#KeyToolJarSigner

When using smartcards, apksigner does not use the same options has Java/keytool/jarsigner (-providerName, -providerClass, -providerArg, -storetype). apksigner documents the options as –ks-provider-class and –ks-provider-arg. Those seem to be accepted but fail when actually making a signature with weird internal exceptions. We use the options that actually work. From: https://geoffreymetais.github.io/code/key-signing/#scripting

fdroidserver.common.signer_fingerprint(cert_encoded)¶

Obtain sha256 signing-key fingerprint for pkcs7 DER certificate.

Extracts hexadecimal sha256 signing-key fingerprint string for a given pkcs7 signature.

Parameters:

Contents of an APK signature.

Returns:

shortened signature fingerprint.

fdroidserver.common.signer_fingerprint_short(cert_encoded)¶

Obtain shortened sha256 signing-key fingerprint for pkcs7 DER certficate.

Extracts the first 7 hexadecimal digits of sha256 signing-key fingerprint for a given pkcs7 signature.

Parameters:

cert_encoded: Contents of an APK signing certificate.

Returns:

shortened signing-key fingerprint.

fdroidserver.common.string_is_integer(string)¶

fdroidserver.common.test_aapt_version(aapt)¶: Check whether the version of aapt is new enough.

fdroidserver.common.test_sdk_exists(thisconfig)¶

fdroidserver.common.unescape_string(string)¶

fdroidserver.common.use_androguard()¶: Report if androguard is available, and config its debug logging.

class fdroidserver.common.vcs(remote, local)¶

Bases: object

Methods

`deinitsubmodules`()
`getref`([revname])	Get current commit reference (hash, revision, etc).
`getsrclib`()	Return the srclib (name, path) used in setting up the current revision, or None.
`gettags`()
`gotorevision`(rev[, refresh])	Take the local repository to a clean version of the given revision.
`gotorevisionx`(rev)	No summary.
`initsubmodules`()
`latesttags`()	Get a list of all the known tags, sorted from newest to oldest.

clientversion
clientversioncmd
repotype

clientversion()¶

clientversioncmd()¶

deinitsubmodules()¶

getref(revname=None)¶: Get current commit reference (hash, revision, etc).

getsrclib()¶: Return the srclib (name, path) used in setting up the current revision, or None.

gettags()¶

gotorevision(rev, refresh=True)¶

Take the local repository to a clean version of the given revision.

Take the local repository to a clean version of the given revision, which is specificed in the VCS’s native format. Beforehand, the repository can be dirty, or even non-existent. If the repository does already exist locally, it will be updated from the origin, but only once in the lifetime of the vcs object. None is acceptable for ‘rev’ if you know you are cloning a clean copy of the repo - otherwise it must specify a valid revision.

gotorevisionx(rev)¶

No summary.

Derived classes need to implement this.

It’s called once basic checking has been performed.

initsubmodules()¶

latesttags()¶: Get a list of all the known tags, sorted from newest to oldest.

repotype()¶

class fdroidserver.common.vcs_bzr(remote, local)¶

Bases: vcs

Methods

`bzr`(args[, envs, cwd, output])	Prevent bzr from ever using SSH to avoid security vulns.
`deinitsubmodules`()
`getref`([revname])	Get current commit reference (hash, revision, etc).
`getsrclib`()	Return the srclib (name, path) used in setting up the current revision, or None.
`gettags`()
`gotorevision`(rev[, refresh])	Take the local repository to a clean version of the given revision.
`gotorevisionx`(rev)	No summary.
`initsubmodules`()
`latesttags`()	Get a list of all the known tags, sorted from newest to oldest.

clientversion
clientversioncmd
repotype

bzr(args, envs={}, cwd=None, output=True)¶: Prevent bzr from ever using SSH to avoid security vulns.

clientversioncmd()¶

gotorevisionx(rev)¶

No summary.

Derived classes need to implement this.

It’s called once basic checking has been performed.

repotype()¶

class fdroidserver.common.vcs_git(remote, local)¶

Bases: vcs

Methods

`checkrepo`()	No summary.
`getref`([revname])	Get current commit reference (hash, revision, etc).
`getsrclib`()	Return the srclib (name, path) used in setting up the current revision, or None.
`gettags`()
`git`(args[, envs, cwd, output])	Prevent git fetch/clone/submodule from hanging at the username/password prompt.
`gotorevision`(rev[, refresh])	Take the local repository to a clean version of the given revision.
`gotorevisionx`(rev)	No summary.
`latesttags`()	Return a list of latest tags.

clientversion
clientversioncmd
deinitsubmodules
initsubmodules
repotype

checkrepo()¶

No summary.

If the local directory exists, but is somehow not a git repository, git will traverse up the directory tree until it finds one that is (i.e. fdroidserver) and then we’ll proceed to destroy it! This is called as a safety check.

clientversioncmd()¶

deinitsubmodules()¶

getref(revname='HEAD')¶: Get current commit reference (hash, revision, etc).

git(args, envs={}, cwd=None, output=True)¶

Prevent git fetch/clone/submodule from hanging at the username/password prompt.

While fetch/pull/clone respect the command line option flags, it seems that submodule commands do not. They do seem to follow whatever is in env vars, if the version of git is new enough. So we just throw the kitchen sink at it to see what sticks.

Also, because of CVE-2017-1000117, block all SSH URLs.

gotorevisionx(rev)¶

No summary.

Derived classes need to implement this.

It’s called once basic checking has been performed.

initsubmodules()¶

latesttags()¶: Return a list of latest tags.

repotype()¶

class fdroidserver.common.vcs_gitsvn(remote, local)¶

Bases: vcs

Methods

`checkrepo`()	No summary.
`deinitsubmodules`()
`getref`([revname])	Get current commit reference (hash, revision, etc).
`getsrclib`()	Return the srclib (name, path) used in setting up the current revision, or None.
`gettags`()
`git`(args[, envs, cwd, output])	Prevent git fetch/clone/submodule from hanging at the username/password prompt.
`gotorevision`(rev[, refresh])	Take the local repository to a clean version of the given revision.
`gotorevisionx`(rev)	No summary.
`initsubmodules`()
`latesttags`()	Get a list of all the known tags, sorted from newest to oldest.

clientversion
clientversioncmd
repotype

checkrepo()¶

No summary.

clientversioncmd()¶

getref(revname='HEAD')¶: Get current commit reference (hash, revision, etc).

git(args, envs={}, cwd=None, output=True)¶

Prevent git fetch/clone/submodule from hanging at the username/password prompt.

AskPass is set to /bin/true to let the process try to connect without a username/password.

The SSH command is set to /bin/false to block all SSH URLs (supported in git >= 2.3). This protects against CVE-2017-1000117.

gotorevisionx(rev)¶

No summary.

Derived classes need to implement this.

It’s called once basic checking has been performed.

repotype()¶

class fdroidserver.common.vcs_hg(remote, local)¶

Bases: vcs

Methods

`deinitsubmodules`()
`getref`([revname])	Get current commit reference (hash, revision, etc).
`getsrclib`()	Return the srclib (name, path) used in setting up the current revision, or None.
`gettags`()
`gotorevision`(rev[, refresh])	Take the local repository to a clean version of the given revision.
`gotorevisionx`(rev)	No summary.
`initsubmodules`()
`latesttags`()	Get a list of all the known tags, sorted from newest to oldest.

clientversion
clientversioncmd
repotype

clientversioncmd()¶

gotorevisionx(rev)¶

No summary.

Derived classes need to implement this.

It’s called once basic checking has been performed.

repotype()¶

fdroidserver.common.vcsearch_g(/, string, pos=0, endpos=sys.maxsize)¶

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

fdroidserver.common.verify_apk_signature(apk, min_sdk_version=None)¶

Verify the signature on an APK.

Try to use apksigner whenever possible since jarsigner is very shitty: unsigned APKs pass as “verified”! Warning, this does not work on JARs with apksigner >= 0.7 (build-tools 26.0.1)

Returns:

Boolean: whether the APK was verified

fdroidserver.common.verify_apks(signed_apk, unsigned_apk, tmp_dir, v1_only=None)¶

Verify that two apks are the same.

One of the inputs is signed, the other is unsigned. The signature metadata is transferred from the signed to the unsigned apk, and then apksigner is used to verify that the signature from the signed APK is also valid for the unsigned one. If the APK given as unsigned actually does have a signature, it will be stripped out and ignored.

Parameters:

signed_apk: Path to a signed APK file
unsigned_apk: Path to an unsigned APK file expected to match it
tmp_dir: Path to directory for temporary files
v1_only: True for v1-only signatures, False for v1 and v2 signatures, or None for autodetection

Returns:

None if the verification is successful, otherwise a string describing what went wrong.

fdroidserver.common.verify_deprecated_jar_signature(jar)¶

Verify the signature of a given JAR file, allowing deprecated algorithms.

index.jar (v0) and index-v1.jar are both signed by MD5/SHA1 by definition, so this method provides a way to verify those. Also, apksigner has different deprecation rules than jarsigner, so this is our current hack to try to represent the apksigner rules when executing jarsigner.

jarsigner is very shitty: unsigned JARs pass as “verified”! So this has to turn on -strict then check for result 4, since this does not expect the signature to be from a CA-signed certificate.

Also used to verify the signature on an archived APK, supporting deprecated algorithms.

F-Droid aims to keep every single binary that it ever published. Therefore, it needs to be able to verify APK signatures that include deprecated/removed algorithms. For example, jarsigner treats an MD5 signature as unsigned.

jarsigner passes unsigned APKs as “verified”! So this has to turn on -strict then check for result 4.

Just to be safe, this never reuses the file, and locks down the file permissions while in use. That should prevent a bad actor from changing the settings during operation.

Raises:

VerificationException: If the JAR’s signature could not be verified.

fdroidserver.common.verify_jar_signature(jar)¶

Verify the signature of a given JAR file.

jarsigner is very shitty: unsigned JARs pass as “verified”! So this has to turn on -strict then check for result 4, since this does not expect the signature to be from a CA-signed certificate.

Raises:

VerificationException: If the JAR’s signature could not be verified.

fdroidserver.common.version_code_string_to_int(vercode)¶: Convert an version code string of any base into an int.

fdroidserver.common.vnsearch_g(/, string, pos=0, endpos=sys.maxsize)¶

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

fdroidserver.common.vnssearch_g(/, string, pos=0, endpos=sys.maxsize)¶

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

fdroidserver.common.write_running_status_json(output)¶

fdroidserver.common.write_status_json(output, pretty=False, name=None)¶: Write status out as JSON, and rsync it to the repo server.

fdroidserver.common.write_to_config(thisconfig, key, value=None, config_file=None)¶

Write a key/value to the local config.yml or config.py.

NOTE: only supports writing string variables.

Parameters:

thisconfig: config dictionary
key: variable name in config to be overwritten/added
value: optional value to be written, instead of fetched from ‘thisconfig’ dictionary.