d3tales_api.Processors package

Submodules

d3tales_api.Processors.back2front module

class d3tales_api.Processors.back2front.CV2Front(id_list=None, backend_data=None, metadata_dict=None, mol_id=None, e_half_scan_rate=0.1, run_anodic=False, run_processing=True, insert=True, redox_event='oxidation', verbose=1, backend_db='backend')

Bases: object

Update frontend db with backend db data for a particular set of data. Copyright 2021, University of Kentucky

__init__(id_list=None, backend_data=None, metadata_dict=None, mol_id=None, e_half_scan_rate=0.1, run_anodic=False, run_processing=True, insert=True, redox_event='oxidation', verbose=1, backend_db='backend')

Parameters

• id_list – list, list of backend ids

• backend_data – list, list of JSON containing processed backend data

• metadata_dict – dict, default metadata dictionary

• mol_id – str, default molecule ID for data if none found

• e_half_scan_rate – float, scan rate at which to extract e half

• run_anodic – bool, run processing for anodic sweep if True

• run_processing – bool, run processing (takes a few minutes) if True

• insert – bool, insert generated data to the frontend D3TaLES database if True

• verbose – int, level of verbosity

• verbose – str, backend DB key word

cv_meta_calcs(electron_num=1, curve_type='anodic')

property e_halfs

classmethod from_json(json_path, **kwargs)

Generate data class from data dict

Parameters

json_path – str, path to JSON file

Returns

data class

static generate_hash(hash_dict)

get_conditions()

process()

prop_dict(value, unit='', order=1, conditions=None, hashes=None, notes=None)

Generate descriptor dictionary in accordance with D3TaLES schema

Parameters

• value – data value

• unit – str, unit

• order – int, property order

• conditions – list, property conditions

• hashes – list, property hash ids

• notes – str, property notes

Returns

descriptor dictionary

class d3tales_api.Processors.back2front.DOI2Front(doi=None, backend_data=None, insert=True, nlp_group='Sarkar')

Bases: object

Update frontend db with backend db data for a particular set of data. Copyright 2021, University of Kentucky

__init__(doi=None, backend_data=None, insert=True, nlp_group='Sarkar')

Parameters

• doi – str, molecule ID

• backend_data – dict, calculation data

• insert – bool, insert generated data to the frontend D3TaLES database if True

• nlp_group – str, name of NLP group

check_extr_props(extracted_props)

static cir_convert(mol_name)

classmethod from_json(json_path, **kwargs)

Generate data class from data dict

Parameters

json_path – str, path to JSON file

Returns

data class

get_literature_data(mol_data)

get_mol_id(mol_data, nlp_group)

class d3tales_api.Processors.back2front.Gaus2FrontCharacterization(_id, calculation_type, conditions, charge, data=None, insert=True, all_props=False, rmsd=True)

Bases: object

Update frontend db with backend db data for a particular set of data. Copyright 2021, University of Kentucky

__init__(_id, calculation_type, conditions, charge, data=None, insert=True, all_props=False, rmsd=True)

Parameters

• _id – str, molecule ID

• calculation_type – str, calculation type

• conditions – dict, calculation conditions

• charge – int, charge

• data – dict, calculation data

• insert – bool, insert generated data to the frontend D3TaLES database if True

• all_props – bool, calculate all properties for the molecule if True

adiabatic_electron_affinity()

Descriptor dict for the adiabatic electron affinity

adiabatic_electron_affinity_2()

Descriptor dict for the second adiabatic electron affinity

adiabatic_ionization_energy()

Descriptor dict for the adiabatic ionization energy

adiabatic_ionization_energy_2()

Descriptor dict for the second adiabatic ionization energy

electron_reorganization_energy()

Descriptor dict for the electron reorganization energy

find_data(calc_type, solvent=None)

Find calculation data

Parameters

• calc_type – str, calculation type

• solvent – str, solvent

Returns

[data dict, hash ID]

classmethod from_data(processing_data, **kwargs)

Generate data class from data dict

Parameters

processing_data – dict, data dict

Returns

data class

get_all_data()

Get all calculations data for object molecule :return: dictionary containing all backend data for the object molecule

get_data(calculations, disregard_missing=False)

Get all data from a list of calculations

Parameters

• calculations – list, list of calculation names

• disregard_missing – bool, disregard missing jobs if True

Returns

[list of hash IDs, dict of calculation name with associated data]

static hash_id(_id, calc_type, gaus_conditions)

Generate hash ID

Parameters

• _id – str, molecule ID

• calc_type – str, calculation type

• gaus_conditions – dict, calculation conditions

Returns

hole_reorganization_energy()

Descriptor dict for the hole reorganization energy

oxidation_potential(electrode='standard_hydrogen_electrode', num_electrons=1)

Descriptor dict for the oxidation potential

reduction_potential(electrode='standard_hydrogen_electrode', num_electrons=1)

Descriptor dict for the reduction potential

relaxation_anion1_groundState()

Descriptor dict for the relaxation energy of the ground -1 anion to the ground state geometry

relaxation_cation1_groundState()

Descriptor dict for the relaxation energy of the +1 cation geometry to the ground state geometry

relaxation_groundState_anion1()

Descriptor dict for the relaxation energy of the ground state geometry to the -1 anion geometry

relaxation_groundState_cation1()

Descriptor dict for the relaxation energy of the ground state geometry to the +1 cation geometry

return_descriptor_dict(value, unit='', hashes=None, name='', order=1, condition_addition=None, **kwargs)

Generate descriptor dictionary in accordance with D3TaLES schema

Parameters

• value – data value

• unit – str, unit

• hashes – list, hash ids

• name – str, property name

• order – int, property order

• condition_addition – dict, additional data to add to conditions

Returns

descriptor dictionary

rmsd_anion1_anion2()

Descriptor dict for the RMSD between the -1 anion and -2 anion geometries

rmsd_cation1_cation2()

Descriptor dict for the RMSD between the +1 cation and +2 cation geometries

rmsd_groundState_anion1()

Descriptor dict for the RMSD between the ground state and -1 anion geometries

rmsd_groundState_cation1()

Descriptor dict for the RMSD between the ground state and +1 cation geometries

solvation_energy(species)

Descriptor dict for the solvation energy

property species_descriptors

Descriptor dict for species descriptors

vertical_electron_affinity()

Descriptor dict for the

vertical_electron_affinity_2()

Descriptor dict for the

vertical_ionization_energy()

Descriptor dict for the vertical ionization energy

vertical_ionization_energy_2()

Descriptor dict for the vertical ionization energy

d3tales_api.Processors.d3tales_parser module

class d3tales_api.Processors.d3tales_parser.ProcessCA(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCA'>, **kwargs)

Bases: ProcessPotBase

Class to process CA data Copyright 2021, University of Kentucky

__init__(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCA'>, **kwargs)

Parameters

• filepath – str, filepath to

• _id – str, molecule ID

• submission_info – dict, submission info

• metadata – dict, metadata

• parsing_class – class, class to use for file parsing (EX: ParseChiCA)

property data_dict

Dictionary of processed data (in accordance with D3TalES backend schema)

class d3tales_api.Processors.d3tales_parser.ProcessCV(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCV'>, **kwargs)

Bases: ProcessPotBase

Class to process CV data Copyright 2021, University of Kentucky

__init__(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCV'>, **kwargs)

Parameters

• filepath – str, filepath to

• _id – str, molecule ID

• submission_info – dict, submission info

• metadata – dict, metadata

• parsing_class – class, class to use for file parsing (EX: ParseChiCV)

property data_dict

Dictionary of processed data (in accordance with D3TalES backend schema)

class d3tales_api.Processors.d3tales_parser.ProcessDFT(_id: ~typing.Optional[str] = None, filepath: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_dft.ProcessGausLog'>)

Bases: object

Class to process molecular DFT output files. Copyright 2021, University of Kentucky

__init__(_id: ~typing.Optional[str] = None, filepath: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_dft.ProcessGausLog'>)

Parameters

• _id – str, molecule ID

• submission_info – dict, submission information

• metadata – dict, metadata (mol_file should be a key to the filepath of the file to be processed)

• parsing_class – class, class to use for file parsing (EX: ProcessGausLog, ProcessCCLIB, or ProcessPsi4Log)

property conditions

Dictionary of conditions (in accordance with D3TaLES backend schema)

property data_dict

Dictionary of processed data (in accordance with D3TalES backend schema)

property hash_id

Hash ID

property is_groundState

True if current species is the ground state, else False

property mol_info

Dictionary containing basic molecule information using the ID from the D3TalES database

class d3tales_api.Processors.d3tales_parser.ProcessNlp(doi=None, instance=None, els_apikey='3a5b9e26201041e25eee70953c65782c', article_download=False, download_dir='temp/')

Bases: object

Class to process NLP data for backend database Copyright 2021, University of Kentucky

__init__(doi=None, instance=None, els_apikey='3a5b9e26201041e25eee70953c65782c', article_download=False, download_dir='temp/')

Parameters

• doi – str, DOI or scopus id

• instance – dict, NLP data. Instance data will override web-scrapped data.

• els_apikey – str, API access key for api.elsevier.com

• article_download – bool, download article and insert article main text if True

• els_apikey – str, directory path into which to download article

property data_dict

download_article()

classmethod from_dataframe(nlp_df, nlp_model, doi=None, base_instance=None, date_generated=datetime.datetime(2024, 2, 10, 8, 47, 23, 21362), **kwargs)

Translate pandas DataFrame from NLP extraction to JSON data conforming to the D3TaLES backend NLP schema: https://github.com/D3TaLES/schema/blob/main/schema_backend/nlp.schema.json. Note that the DataFrame should contain the all the columns listed in the expected_columns variable, and no other columns.

Parameters

• nlp_df – pandas DataFrame, extracted NLP data

• nlp_model – str, name of NLP model used for data extraction

• doi – str, DOI associated with extracted data

• base_instance – dict, base instance containing additional info for backend data insertion

• date_generated – datetime obj, date and time the data was generated

Returns

dict, JSON formatted data conforming to the D3TaLES backend NLP schema

classmethod from_html(html_txt, nlp_model, doi=None, base_instance=None, date_generated=datetime.datetime(2024, 2, 10, 8, 47, 23, 21364), **kwargs)

classmethod from_parsed_json(json_path, **kwargs)

Generate data class from JSON file

Parameters

json_path – str, path to JSON file

Returns

data class

classmethod from_prop_list(prop_list, nlp_model, doi=None, base_instance=None, date_generated=datetime.datetime(2024, 2, 10, 8, 47, 23, 21365), **kwargs)

static get_affiliations(affiliation)

get_basic_info()

Dictionary of processed data (in accordance with D3TalES backend schema)

get_main_text()

make_author_entry(author)

class d3tales_api.Processors.d3tales_parser.ProcessPotBase(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCV'>, **kwargs)

Bases: object

Base class for processing potentiostat files Copyright 2021, University of Kentucky

__init__(filepath, _id: ~typing.Optional[str] = None, submission_info: ~typing.Optional[dict] = None, metadata: ~typing.Optional[dict] = None, parsing_class=<class 'd3tales_api.Processors.parser_cv.ParseChiCV'>, **kwargs)

Parameters

• filepath – str, filepath to

• _id – str, molecule ID

• submission_info – dict, submission info

• metadata – dict, metadata

• parsing_class – class, class to use for file parsing (EX: ParseChiCV)

property data_dict

Dictionary of processed data (in accordance with D3TalES backend schema)

static data_format(data)

Format measurement data

Parameters

data – measurement data

Returns

formatted measurement data, EX: {“value”: 0.5, “unit”: “V”}

property mol_info

Dictionary containing basic molecule information using the ID from the D3TalES database

property pot_conditions

Dictionary of conditions (in accordance with D3TaLES backend schema)

static reagent_format(reagents, purity=None)

Convert reagent data to D3TaLES schema format

Parameters

• reagents – reagent data or list of reagent data

• purity – default purity value

Returns

formatted reagent data

class d3tales_api.Processors.d3tales_parser.ProcessUvVis(filepath, mol_id, metadata=None, parsing_class=<class 'd3tales_api.Processors.parser_uvvis.ParseExcel'>)

Bases: object

Class to process UV-Vis data files. Copyright 2021, University of Kentucky

__init__(filepath, mol_id, metadata=None, parsing_class=<class 'd3tales_api.Processors.parser_uvvis.ParseExcel'>)

Parameters

• filepath – str, filepath to data file

• mol_id – str, molecule ID

• metadata – dict, dictionary containing any metadata for this molecule, e.g., {“solvent”: “acetonitrile”}

• parsing_class – class, a UV-Vis parsing class with which to parse the data

property no_sql_data

UV-Vis information in a dictionary that matches the No-SQL schema

property sql_absorbance_data

UV-Vis information in a dictionary that matches the SQL AbsorbanceData Table schema

property sql_data

UV-Vis information in a dictionary that matches the SQL UVVisData Table schema

d3tales_api.Processors.d3tales_parser.float_from_str(text)

d3tales_api.Processors.expflow_parser module

class d3tales_api.Processors.expflow_parser.ProcessExpFlowObj(expflow_obj, source_group, **kwargs)

Bases: object

Base class for Processing ExpFlow objects for data analysis

__init__(expflow_obj, source_group, **kwargs)

Parameters

• expflow_obj – obj or dict, ExpFlow object to process

• source_group – str, source group for ExpFlow object

• kwargs – object for processing ExpFlow objects

property apparatus_by_id

Dictionary of apparatus ids and apparatus names

get_apparatus(apparatus_type, get_uuid=False)

Get apparatus by type NOTE: this only works if there is only one apparatus of the given type

Parameters

• apparatus_type – str, type of apparatus to get

• get_uuid – bool, return uuid of apparatus, else return apparatus name

Returns

apparatus name

get_concentration(solvent_uuids, redox_uuid, beaker_uuid)

Set temporary properties for concentration calculator

Parameters

• solvent_uuids – list, list of solvent uuids

• redox_uuid – str, uuid for redox-active molecule

• beaker_uuid – str, uuid for beaker

Returns

concentration data dict

static get_param_quantity(param_list)

Get parameters with mass and/or volume quantities for all parameters in a list of parameters

Parameters

param_list – list of parameters

Returns

list of parameters with mass and/or volume quantities

get_reagents(reagent_type, get_uuids=False)

Get all reagents of a given type

Parameters

• reagent_type – str, type of reagent to get

• get_uuids – bool, return uuid of electrode, else return electrode name

Returns

list of reagents (as dict, EX: [{“name”: water, “purity”: 0.95], or as uuids)

property instrument_name

Instrument name

property molecule_id

Molecule ID

property_by_action(action_names, parameter_idx=0)

Get molecule_id working electrode surface area NOTE: this only works if there is only one action of the given name

Parameters

• action_names – str or list, name of action to get

• parameter_idx – index of parameter to get if more than one

Returns

return property as data dict

property reagents_by_id

Dictionary of reagent ids and reagent names

property redox_mol

Redox molecule instance

property redox_mol_concentration

Concentration of redox-active molecule

property solvent

Redox molecule instance

class d3tales_api.Processors.expflow_parser.ProcessExperimentRun(expflow_obj, source_group, **kwargs)

Bases: ProcessExpFlowObj

Class for Processing Experiment Run object for data analysis

property cv_metadata

Dictionary of CV metadata (in accordance with D3TaLES schema)

get_electrode(electrode, get_uuid=False)

Get electrode from experiment data

Parameters

• electrode – str,which electrode (working, counter, or reference)

• get_uuid – bool, return uuid of electrode, else return electrode name

Returns

electrode (uuid or name)

d3tales_api.Processors.expflow_parser.get_id(o)

Get id (either _id or uuid) from a data object

d3tales_api.Processors.parser_cv module

class d3tales_api.Processors.parser_cv.ParseChiBase(file_path, data_header='Potential', delimiter=',')

Bases: object

Extract data from raw Chi experiment files

__init__(file_path, data_header='Potential', delimiter=',')

Parameters

file_path – str, filepath to experiment data file

static extract_value_unit(line, value_break='=', value_type='str', return_dict=False)

Extract data value and unit from a Chi CV data line

Parameters

• line – str, data line

• value_break – str, type of character which breaks the line

• value_type – str, value datatype

• return_dict – bool, return data dict if True

Returns

value or data dict

class d3tales_api.Processors.parser_cv.ParseChiCA(file_path, data_header='Time', delimiter=',')

Bases: ParseChiBase

Extract data from raw Chi CV experiment files

__init__(file_path, data_header='Time', delimiter=',')

Parameters

file_path – str, filepath to experiment data file

get_data_calcs(calc_name='Slp', header='Forward')

property resistance

class d3tales_api.Processors.parser_cv.ParseChiCV(file_path, data_header='Potential', delimiter=',', min_scan_len=5)

Bases: ParseChiBase

Extract data from raw Chi CV experiment files

__init__(file_path, data_header='Potential', delimiter=',', min_scan_len=5)

Parameters

file_path – str, filepath to experiment data file

calculate_plotting(prop_name, return_type=<class 'dict'>)

Calculate a given plotting property using the D3TaLES Plotters

Parameters

• prop_name – str, property name

• return_type – str, datatype to return

Returns

plotting property or empty datatype

calculate_prop(prop_name, return_type=<class 'dict'>)

Calculate a given property using the D3TaLES CV Calculators

Parameters

• prop_name – str, property name

• return_type – str, datatype to return

Returns

property or empty datatype

class d3tales_api.Processors.parser_cv.ParseChiESI(file_path, data_header='Freq', delimiter=',')

Bases: ParseChiBase

Extract data from raw Chi CV experiment files

__init__(file_path, data_header='Freq', delimiter=',')

Parameters

file_path – str, filepath to experiment data file

d3tales_api.Processors.parser_cv.break_list(lst, return_brks=False)

d3tales_api.Processors.parser_cv.break_scan_data(scan_data)

d3tales_api.Processors.parser_dft module

class d3tales_api.Processors.parser_dft.ProcessCCLIB(metadata=None)

Bases: object

Class to process most logfiles using https://cclib.github.io/index.html. Copyright 2021, University of Kentucky

__init__(metadata=None)

Parameters

metadata – dict, metadata (mol_file should be a key to the filepath of the file to be processed)

cclib_parse()

Use CCLIB to parse data file

static get_freq_dicts(cmol)

Get a pymatgen frequency dictionary list from a cclib molecule object :return: list of frequency dicts

get_radical_stability_score(spin_type='mulliken')

Get radical stability score :param spin_type: str, type of spin to use :return: dictionary of radical stability score and its components

static get_sites_dict(cmol)

Get a pymatgen sites dictionary list from a cclib molecule object :return: list of atom site dicts

class d3tales_api.Processors.parser_dft.ProcessGausLog(metadata=None)

Bases: ProcessCCLIB

Class to process Gaussian logfiles. Copyright 2021, University of Kentucky

__init__(metadata=None)

Parameters

metadata – dict, metadata (mol_file should be a key to the filepath of the file to be processed)

property dipole_moments

A list of tuples, each containing teh (X, Y, Z) coordinate for a dipole moment such as [(X1, Y1, Z1), (X2, Y2, Z2)… ]

static get_pmgmol(file)

Get Pymatgen molecule object from file

Parameters

file – str, file path

Returns

Pymatgen molecule object

get_solvent_info()

Get solvent information

static get_tddft_excitations(log_path)

Read excitation energies after a TD-DFT calculation.

Parameters

log_path – str, filepath to log file

Returns

A list of tuple for each transition such as [(energy (eV), lambda (nm), oscillatory strength), … ]

property homo_lumo

HOMO and LUMO energies from a Gaussian molecule: [homo, lumo] - list containing homo then lumo in eV

property omega

Omega value from w-tuning output file (generated with the OCELOT w-tuning module)

parse_tddft(sigma=0.1, step=0.01)

Parse TDDFT data

Parameters

• sigma –

• step –

property pcm_info

Solvent and dielectric constant from Gaussian logfile: [solvent_name, dielectric_constant] - list containing solvent name and dielectric constant]

property runtime

Runtime in core hours from a logfile

property tuning_parameter

Omega value from Pymatgen molecule object

class d3tales_api.Processors.parser_dft.ProcessPsi4Log(metadata=None)

Bases: ProcessCCLIB

Class to process Psi4 logfiles. Copyright 2021, University of Kentucky

__init__(metadata=None)

Parameters

metadata – dict, metadata (mol_file should be a key to the filepath of the file to be processed)

d3tales_api.Processors.parser_ir module

d3tales_api.Processors.parser_uvvis module

class d3tales_api.Processors.parser_uvvis.ParseExcel(filepath)

Bases: object

Class to process raw UV-Vis data in an Excel format Copyright 2021, University of Kentucky

__init__(filepath)

Parameters

filepath – str, filepath to Excel data file

property absorbance_data

Absorbance data (dict)

property date_recorded

Data Recorded

property integration_time

Integration time

parse_file()

Use Pandas to parse the raw data file

d3tales_api.Processors.processing_firetasks module

class d3tales_api.Processors.processing_firetasks.ApproveJobs(*args, **kwargs)

Bases: FiretaskBase

Approve multiple processing jobs

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.FileTransferTask(*args, **kwargs)

Bases: FiretaskBase

A Firetask to Transfer files. Note that

Parameters

• mode – str, - move, mv, copy, cv, copy2, copytree, copyfile, rtransfer

• files – ([str]) or ([(str, str)]) - list of source files, or dictionary containing ‘src’ and ‘dest’ keys

• dest – str, optional, destination directory, if not specified within files parameter (else optional)

• server – str, optional, server host for remote transfer

• user – str, optional, user to authenticate with on remote server

• key_filename – str, optional, optional SSH key location for remote transfer

• max_retry – int, optional, number of times to retry failed transfers; defaults to 0 (no retries)

• retry_delay – int, optional, number of seconds to wait between retries; defaults to 10

• ignore_errors – bool, optional, Whether to ignore errors. Defaults to False.

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.ProcessFile(*args, **kwargs)

Bases: FiretaskBase

Process a data File

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.SendToStorage(*args, **kwargs)

Bases: FiretaskBase

Send data file to D3TaLES storage

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.UpdateApprovalStatus(*args, **kwargs)

Bases: FiretaskBase

Update the backend D3TaLES database approved status

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.UpdateBackendDB(*args, **kwargs)

Bases: FiretaskBase

Update the backend D3TaLES database with parsed data

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.UpdateFrontendDB(*args, **kwargs)

Bases: FiretaskBase

Update the frontend D3TaLES database with parsed data

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

class d3tales_api.Processors.processing_firetasks.UpdateUserDB(*args, **kwargs)

Bases: FiretaskBase

Update the D3TaLES website user database approved status

run_task(fw_spec)

This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

Args:

fw_spec (dict): A Firework spec. This comes from the master spec.

In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

Returns:

(FWAction)

d3tales_api.Processors.processing_firetasks.mkdir_p(sftp, remote_directory)

d3tales_api.Processors.processing_firetasks.test_path(sftp, destination_path)

Module contents