plate_model_manager.utils package

Submodules

plate_model_manager.utils.collect_update_model module

collect_model(model_name, target_dir='.', source='https://repo.gplates.org/webdav/pmm/config/model_sources.json')

create_hex_hash_sidecar_files(model_path)

Create SHA256 hash sidecar files for all .zip files in the given directory.

upload_model(model_path, remote_target, identity_file)

Upload a plate model folder to a remote server via SCP.

This function uploads all files in the specified model directory to a remote server. Before uploading, it creates SHA256 hash sidecar files for all .zip files in the directory. Existing files on the remote server are archived with a timestamp before the new files are uploaded.

Parameters:

• model_path – Path to the local model directory containing files to upload.

• remote_target – Remote destination in the format user@host:path. The model name is appended to the path automatically.

• identity_file – Path to the SSH private key file for authentication.

Raises:

• RuntimeError – If no files are found in the model directory or if the SSH connection to the remote host fails.

• AssertionError – If remote_target is invalid and does not contain :.

Example:

upload_model(
    "models/muller2022",
    "ubuntu@example.com:/data/models",
    "~/.ssh/id_rsa",
)

plate_model_manager.utils.download module

class FileDownloader(file_url: str, meta_filepath: str, dst_dir: str, filename: str | None = None, auto_unzip: bool = True, expire_hours=12, expiry_time_format='%Y/%m/%d, %H:%M:%S', large_file_hint=False, timeout=(None, None), http_client: HttpClient = HttpClient.REQUESTS)

Bases: object

Class for managing single file download

check_if_expire_date_need_update()

Return whether only the metadata expiry timestamp should be refreshed.

This helper is typically called after check_if_file_need_update() has fetched remote state. It returns True when remote content is unchanged and therefore the local file can be kept while extending the metadata expiry value.

Match conditions (either is sufficient):

• SHA-256 path: self.new_sha256 is available and equals self.meta_sha256.

• ETag fallback path: self.new_etag is available and equals self.meta_etag.

Returns:

True if content identity is unchanged and expiry metadata should be updated, otherwise False.

Return type:

bool

check_if_file_need_update()

Decide whether the target file should be downloaded again.

Returns:

True when the file should be downloaded/re-downloaded, False when the existing local file can be reused.

Return type:

bool

Decision flow:

1. If the metadata file is missing, return True.

2. If the stored url differs from self.file_url (or is missing), return True.

3. If the metadata expiry is still valid, return False.

4. If expired (or expiry is invalid/missing), compare remote content state: - Prefer SHA-256 comparison when available. - Fall back to ETag comparison if SHA-256 cannot be obtained. - If neither reliable value is available, return True.

download_file_and_update_metadata()

Download the target file and refresh its metadata cache.

The method selects an HTTP backend from self.http_client and then chooses download strategy based on size:

• If self.large_file_hint is True, it first retrieves response headers and stores self.file_size.

• If self.file_size is known and greater than 20 MB, it uses fetch_large_file.

• Otherwise, it uses fetch_file.

After a successful download call, self.new_etag is updated from the client response and update_metadata() is called to write metadata fields (URL, expiry, ETag, SHA-256) to self.meta_filepath.

Raises:

Exception – Propagates exceptions raised by network/header retrieval, download client calls, or metadata writing.

update_metadata()

Write or refresh the JSON metadata file for the current download.

The metadata file at self.meta_filepath is created (including parent directories) and overwritten with these fields:

• url: self.file_url

• expiry: current time plus self.expire_hours formatted with self.expiry_time_format

• etag: self.new_etag

• sha256: self.new_sha256

If self.new_sha256 is not already populated, SHA-256 is retrieved from the remote resource before writing metadata.

Raises:

Exception – Propagates exceptions raised while fetching SHA-256, creating directories, or writing the metadata file.

class HttpClient(*values)

Bases: Enum

AIOHTTP = 2

REQUESTS = 1

plate_model_manager.utils.enums module

class GenerationMethod(*values)

Bases: Enum

Isochrons = 'UsingIsochrons'

Topologies = 'UsingTopologies'

class ReferenceFrame(*values)

Bases: Enum

MantleReferenceFrame = 'MantleFrame'

PmagReferenceFrame = 'PMAG'

plate_model_manager.utils.layer_validation module

list_remote_zip_layers(model_name, base_url, timeout=30)

load_models_config(source, timeout=30)

normalize_svr_base_url(all_models)

validate_layers(model_config, base_url=None, timeout=30)

plate_model_manager.utils.misc module

add_logging_file(filename: str = 'pmm.log', level=20)

disable_stdout_logging()

get_distribution_version()

get the version string from the package metadata

is_debug_mode()

Check if the debug mode is enabled by checking the environment variable “PMM_DEBUG”.

export PMM_DEBUG=true to enable the debug mode.

my_warningformat(message, category, filename, lineno, line=None)

print_error(msg)

print_warning(msg)

set_logging_level(level=30)

setup_logging()

turn_on_debug_logging()

plate_model_manager.utils.network module

get_content_length(headers)

get_etag(headers)

return the etag in the headers. The return could be none if the server does not support etag.

Parameters:

headers – call get_headers(url) to get headers

get_headers(url, timeout=(None, None))

get_sha256(url, timeout=(None, None))

Return the sha256 hash from the sidecar file name in a WebDAV/XML or HTML directory listing.

Parameters:

timeout – a (connect_timeout, read_timeout) tuple for requests.get(). None keeps requests’ default behavior for that timeout component.

plate_model_manager.utils.unzip module

save_compressed_data(url, data, dst_path)

extract files from compressed data

Parameters:

• url – URL

• data – bytes-like object

• dst_path – location to save the files

Module contents

Utility helpers for plate_model_manager.