Utilities
Various utilities to help APS use the Bluesky framework.
Also consult the Index under the apstools heading for links to the Exceptions, and Utilities described here.
Utilities by Activity
Finding
|
Find the ophyd (dotted name) object associated with the given ophyd name. |
|
Find all ophyd objects associated with the given EPICS PV. |
|
Listing
|
Describe the signal information from device |
|
Show all the ophyd Signal and Device objects defined as globals. |
|
List all plans. |
|
Convenience function to list all keys (column names) in the scan's stream (default: primary). |
|
List the runs from the given catalog according to some options. |
|
List runs from catalog. |
Reporting
|
custom print the RunEngine metadata in a table |
|
print (stdout) a list of all snapshots in the databroker |
|
Slit size and center as a named tuple |
Other Utilities
|
convert text so it can be used as a dictionary key |
|
Given list of EPICS PV names, return dict of EpicsSignal objects. |
|
send email notifications when requested |
|
Get the first live plot that matches |
|
Get the MatPlotLib Figure window for y vs x. |
|
Find the plot(s) by name and replot with at most the last n lines. |
|
Find the plot with axes x and y and replot with at most the last n lines. |
String must not exceed EPICS PV length. |
|
|
Run a UNIX command, returns (stdout, stderr). |
General
|
convert text so it can be used as a dictionary key |
|
format a command list as a pyRestTable.Table object |
|
Given list of EPICS PV names, return dict of EpicsSignal objects. |
|
copy filtered runs from source_cat to target_cat |
|
Searches the databroker v2 database. |
|
return a text table from |
|
send email notifications when requested |
|
base class: read-only support for Excel files, treat them like databases |
|
Generic (read-only) handling of Excel spreadsheet-as-database |
|
Exception when reading Excel spreadsheet. |
|
Find the ophyd (dotted name) object associated with the given ophyd name. |
|
Find all ophyd objects associated with the given EPICS PV. |
|
|
|
Return the full dotted name |
|
|
|
Return Bluesky database using keyword guides or default choice. |
|
|
Find the "default" database (has the most recent run). |
|
|
get the IPython shell's namespace dictionary (or globals() if not found) |
|
Convenience function to get the run's data. |
|
Convenience function to get value of key in run stream. |
|
Get values from a previous scan stream in a databroker catalog. |
return the name of the current ipython profile or |
|
|
Format a list of items. |
|
Describe the signal information from device |
|
Show all the ophyd Signal and Device objects defined as globals. |
|
List all plans. |
|
Convenience function to list all keys (column names) in the scan's stream (default: primary). |
|
List the runs from the given catalog according to some options. |
|
List runs from catalog. |
Define parameters that can be overridden from a user configuration file. |
|
|
break a list (or other iterable) into pairs |
|
custom print the RunEngine metadata in a table |
|
print (stdout) a list of all snapshots in the databroker |
|
Print table of different |
|
Set EPICS motor record's user coordinate to |
|
Replay the document stream from one (or more) scans (headers). |
|
return memory used by this process |
|
(decorator) run |
|
make text safe to be used as an ophyd object name |
|
Get the first live plot that matches |
|
Get the MatPlotLib Figure window for y vs x. |
|
splits a line into words some of which might be quoted |
|
Report bluesky run metrics from the databroker. |
|
Encode |
|
|
|
Find the plot(s) by name and replot with at most the last n lines. |
|
Find the plot with axes x and y and replot with at most the last n lines. |
String must not exceed EPICS PV length. |
|
|
Run a UNIX command, returns (stdout, stderr). |
Submodules
Working with databroker catalogs
|
copy filtered runs from source_cat to target_cat |
|
|
|
|
|
Return Bluesky database using keyword guides or default choice. |
|
|
Find the "default" database (has the most recent run). |
|
|
Get values from a previous scan stream in a databroker catalog. |
|
Print table of different |
- apstools.utils.catalog.copy_filtered_catalog(source_cat, target_cat, query=None)[source]
copy filtered runs from source_cat to target_cat
PARAMETERS
- source_cat
obj : instance of databroker.Broker or databroker.catalog[name]
- target_cat
obj : instance of databroker.Broker or databroker.catalog[name]
- query
dict : mongo query dictionary, used to filter the results (default:
{})see: https://docs.mongodb.com/manual/reference/operator/query/
example:
copy_filtered_catalog( databroker.Broker.named("mongodb_config"), databroker.catalog["test1"], {'plan_name': 'snapshot'})
- apstools.utils.catalog.getDatabase(db=None, catalog_name=None)[source]
Return Bluesky database using keyword guides or default choice.
PARAMETERS
- db
object : Bluesky database, an instance of
databroker.catalog(default: seecatalog_namekeyword argument)- catalog_name
str : Name of databroker v2 catalog, used when supplied
dbisNone. (default: catalog with most recent run timestamp)
RETURNS
- object or
None: Bluesky database, an instance of
databroker.catalog
(new in release 1.4.0)
- apstools.utils.catalog.getDefaultDatabase()[source]
Find the “default” database (has the most recent run).
Note that here, database and catalog mean the same.
This routine looks at all the database instances defined in the current session (console or notebook). If there is only one or no database instances defined as objects in the current session, the choice is simple. When there is more than one database instance in the current session, then the one with the most recent run timestamp is selected. In the case (as happens when starting with a new database) that the current database has no runs and another database instance is defined in the session and that additional database has runs in it (such as the previous database), then the database with the newest run timestamp (and not the newer empty database) will be chosen.
RETURNS
- object or
None: Bluesky database, an instance of
databroker.catalog
(new in release 1.4.0)
- object or
- apstools.utils.catalog.getStreamValues(scan_id, key_fragment='', db=None, stream='baseline', query=None, use_v1=True)[source]
Get values from a previous scan stream in a databroker catalog.
Optionally, select only those data with names including
key_fragment.Tip
If the output is truncated, use
pd.set_option('display.max_rows', 300)to increase the number of rows displayed.PARAMETERS
- scan_id
int or str : Scan (run) identifier. Positive integer value is
scan_idfrom run’s metadata. Negative integer value is since most recent run in databroker. String is run’suidunique identifier (can abbreviate to the first characters needed to assure it is unique).- key_fragment
str : Part or all of key name to be found in selected stream. For instance, if you specify
key_fragment="lakeshore", it will return all the keys that includelakeshore.- db
object : Bluesky database, an instance of
databroker.catalog. Default: will search existing session for instance.- stream
str : Name of the bluesky data stream to obtain the data. Default: ‘baseline’
- query
dict : mongo query dictionary, used to filter the results Default:
{}see: https://docs.mongodb.com/manual/reference/operator/query/
- use_v1
bool : Chooses databroker API version between ‘v1’ or ‘v2’. Default:
True(meaning use the v1 API)
RETURNS
- object :
pandas DataFrame with values from selected stream, search_string, and query
see: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
(new in apstools 1.5.1)
- apstools.utils.catalog.quantify_md_key_use(key=None, db=None, catalog_name=None, since=None, until=None, query=None)[source]
Print table of different
keyvalues and how many times each appears.PARAMETERS
- key str :
one of the metadata keys in a run’s start document (default:
plan_name)- db object :
Instance of databroker v1
Brokeror v2catalog(default: seecatalog_namekeyword argument)- catalog_name str :
Name of databroker v2 catalog, used when supplied
dbisNone. (default:mongodb_config)- since str :
include runs that started on or after this ISO8601 time (default:
1995-01-01)- until str :
include runs that started before this ISO8601 time (default:
2100-12-31)- query dict :
mongo query dictionary, used to filter the results (default:
{})see: https://docs.mongodb.com/manual/reference/operator/query/
EXAMPLES:
quantify_md_key_use(key="proposal_id") quantify_md_key_use(key="plan_name", catalog_name="9idc", since="2020-07") quantify_md_key_use(key="beamline_id", catalog_name="9idc") quantify_md_key_use(key="beamline_id", catalog_name="9idc", query={'plan_name': 'Flyscan'}, since="2020", until="2020-06-21 21:51") quantify_md_key_use(catalog_name="8id", since="2020-01", until="2020-03") In [8]: quantify_md_key_use(catalog_name="apstools_test") ========= ===== plan_name #runs ========= ===== count 26 scan 27 ========= ===== In [9]: quantify_md_key_use(catalog_name="usaxs_test") ========================== ===== plan_name #runs ========================== ===== Flyscan 1 TuneAxis.tune 1 count 1 measure_USAXS_Transmission 1 run_Excel_file 1 snapshot 1 tune_a2rp 1 tune_ar 1 tune_m2rp 1 tune_mr 1 ========================== =====
Device information
|
Describe the signal information from device |
- apstools.utils.device_info.listdevice(obj, scope=None, cname=False, dname=True, show_pv=False, use_datetime=True, show_ancient=True)[source]
Describe the signal information from device
objin a pandas DataFrame.Look through all subcomponents to find all the signals to be shown.
PARAMETERS
- obj
object : Instance of ophyd Signal or Device.
- scope
str or None : Scope of content to be shown.
"full"(orNone) shows all Signal components"epics"shows only EPICS-based Signals"read"shows only the signals returned byobj.read()
default:
None- cname
bool : Show the _control_ (Python, dotted) name in column
name.default:
False- dname
bool : Show the _data_ (databroker, with underlines) name in column
data name.default:
True- show_pv
bool : Show the EPICS process variable (PV) name in column
PV.default:
False- use_datetime
bool : Show the EPICS timestamp (time of last update) in column
timestamp.default:
True- show_ancient
bool : Show uninitialized EPICS process variables.
In EPICS, an uninitialized PV has a timestamp of 1990-01-01 UTC. This option enables or suppresses ancient values identified by timestamp from 1989. These are values only defined in the original
.dbfile.default:
True
email Support
|
send email notifications when requested |
- class apstools.utils.email.EmailNotifications(sender=None)[source]
send email notifications when requested
use default OS mail utility (so no credentials needed)
EXAMPLE
Send email(s) when feedback_limits_approached (a hypothetical boolean) is True:
# setup from apstools.utils import EmailNotifications SENDER_EMAIL = "instrument_user@email.host.tld" email_notices = EmailNotifications(SENDER_EMAIL) email_notices.add_addresses( # This list receives email when send() is called. "joe.user@goodmail.com", "instrument_team@email.host.tld", # others? ) # ... later if feedback_limits_approached: # send emails to list subject = "Feedback problem" message = "Feedback is very close to its limits." email_notices.send(subject, message)
- run_in_thread()
(decorator) run
funcin threadUSAGE:
@run_in_thread def progress_reporting(): logger.debug("progress_reporting is starting") # ... #... progress_reporting() # runs in separate thread #...
Directory of the known plans
|
List all plans. |
- apstools.utils.list_plans.listplans(base=None, trunc=40)[source]
List all plans. (Actually, lists all generator functions).
NOTE: Can only detect generator functions. Bluesky plans are generator functions that generate
bluesky.Msgobjects. There is a PR to define a decorator that identifies a generator function as a bluesky plan.PARAMETERS
- base
object or None : Object that contains plan methods (if
None, use global namespace.) (default:None)- trunc
int : Truncate long docstrings to no more than
trunccharacters. (default: 40)
Directory of bluesky runs
|
Convenience function to get the run's data. |
|
Convenience function to get value of key in run stream. |
|
Convenience function to list all keys (column names) in the scan's stream (default: primary). |
|
List the runs from the given catalog according to some options. |
|
List runs from catalog. |
|
Report bluesky run metrics from the databroker. |
- class apstools.utils.list_runs.ListRuns(cat: Optional[object] = None, query: Optional[dict] = None, keys: Optional[str] = None, missing: str = '', num: int = 20, reverse: bool = True, since: Optional[str] = None, sortby: str = 'time', timefmt: str = '%Y-%m-%d %H:%M:%S', until: Optional[str] = None, ids: Optional[Any] = None)[source]
List the runs from the given catalog according to some options.
EXAMPLE:
ListRuns(cat).to_dataframe()
PUBLIC METHODS
Output as pandas DataFrame object
to_table([fmt])Output as pyRestTable object.
Parse the runs for the given metadata keys.
INTERNAL METHODS
_get_by_key(md, key)Get run's metadata value by key.
_check_cat()_apply_search_filters()Search for runs from the catalog.
_check_keys()Check that self.keys is a list of strings.
- apstools.utils.list_runs.getRunData(scan_id, db=None, stream='primary', query=None, use_v1=True)[source]
Convenience function to get the run’s data. Default is the
primarystream.PARAMETERS
- scan_id
int or str : Scan (run) identifier. Positive integer value is
scan_idfrom run’s metadata. Negative integer value is since most recent run in databroker. String is run’suidunique identifier (can abbreviate to the first characters needed to assure it is unique).- db
object : Bluesky database, an instance of
databroker.catalog. Default: will search existing session for instance.- stream
str : Name of the bluesky data stream to obtain the data. Default: ‘primary’
- query
dict : mongo query dictionary, used to filter the results Default:
{}see: https://docs.mongodb.com/manual/reference/operator/query/
- use_v1
bool : Chooses databroker API version between ‘v1’ or ‘v2’. Default:
True(meaning use the v1 API)
(new in apstools 1.5.1)
- apstools.utils.list_runs.getRunDataValue(scan_id, key, db=None, stream='primary', query=None, idx=- 1, use_v1=True)[source]
Convenience function to get value of key in run stream.
Defaults are last value of key in primary stream.
PARAMETERS
- scan_id
int or str : Scan (run) identifier. Positive integer value is
scan_idfrom run’s metadata. Negative integer value is since most recent run in databroker. String is run’suidunique identifier (can abbreviate to the first characters needed to assure it is unique).- key
str : Name of the key (data column) in the table of the stream’s data. Must match identically.
- db
object : Bluesky database, an instance of
databroker.catalog. Default: will search existing session for instance.- stream
str : Name of the bluesky data stream to obtain the data. Default: ‘primary’
- query
dict : mongo query dictionary, used to filter the results Default:
{}see: https://docs.mongodb.com/manual/reference/operator/query/
- idx
int or str : List index of value to be returned from column of table. Can be
0for first value,-1for last value,"mean"for average value, or"all"for the full list of values. Default:-1- use_v1
bool : Chooses databroker API version between ‘v1’ or ‘v2’. Default:
True(meaning use the v1 API)
(new in apstools 1.5.1)
- apstools.utils.list_runs.listRunKeys(scan_id, key_fragment='', db=None, stream='primary', query=None, strict=False, use_v1=True)[source]
Convenience function to list all keys (column names) in the scan’s stream (default: primary).
PARAMETERS
- scan_id
int or str : Scan (run) identifier. Positive integer value is
scan_idfrom run’s metadata. Negative integer value is since most recent run in databroker. String is run’suidunique identifier (can abbreviate to the first characters needed to assure it is unique).- key_fragment
str : Part or all of key name to be found in selected stream. For instance, if you specify
key_fragment="lakeshore", it will return all the keys that includelakeshore.- db
object : Bluesky database, an instance of
databroker.catalog. Default: will search existing session for instance.- stream
str : Name of the bluesky data stream to obtain the data. Default: ‘primary’
- query
dict : mongo query dictionary, used to filter the results Default:
{}see: https://docs.mongodb.com/manual/reference/operator/query/
- strict
bool : Should the
key_fragmentbe matched identically (strict=True) or matched by lower case comparison (strict=False)? Default:False- use_v1
bool : Chooses databroker API version between ‘v1’ or ‘v2’. Default:
True(meaning use the v1 API)
(new in apstools 1.5.1)
- apstools.utils.list_runs.listruns(cat=None, keys=None, missing='', num=20, printing='smart', reverse=True, since=None, sortby='time', tablefmt='dataframe', timefmt='%Y-%m-%d %H:%M:%S', until=None, ids=None, **query)[source]
List runs from catalog.
This function provides a thin interface to the highly-reconfigurable
ListRuns()class in this package.PARAMETERS
- cat
object : Instance of databroker v1 or v2 catalog.
- keys
str or [str] or None: Include these additional keys from the start document. (default:
Nonemeans"scan_id time plan_name detectors")- missing
str: Test to report when a value is not available. (default:
"")- ids
[int] or [str]: List of
uidorscan_idvalue(s). Can mix different kinds in the same list. Also can specify offsets (e.g.,-1). According to the rules fordatabrokercatalogs, a string is auid(partial representations allowed), an int isscan_idif positive or an offset if negative. (default:None)- num
int : Make the table include the
nummost recent runs. (default:20)- printing
bool or
"smart": IfTrue, print the table to stdout. If"smart", then act as shown below. (default:True)session
action(s)
python session
print and return
NoneIpython console
return
DataFrameobjectJupyter notebook
return
DataFrameobject- reverse
bool : If
True, sort in descending order bysortby. (default:True)- since
str : include runs that started on or after this ISO8601 time (default:
"1995-01-01")- sortby
str : Sort columns by this key, found by exact match in either the
startorstopdocument. (default:"time")- tablefmt
str : When returning an object, specify which type of object to return. (default:
"dataframe",)value
object
dataframe
pandas.DataFrametable
str(pyRestTable.Table)- timefmt
str : The
timekey (also includes keys"start.time"and"stop.time") will be formatted by theself.timefmtvalue. See https://strftime.org/ for examples. The specialtimefmt="raw"is used to report time as the raw value (floating point time as used in python’stime.time()). (default:"%Y-%m-%d %H:%M:%S",)- until
str : include runs that started before this ISO8601 time (default:
2100-12-31)**querydict : Any additional keyword arguments will be passed to the databroker to refine the search for matching runs using the
mongoquerypackage.
RETURNS
- object:
Noneorstrorpd.DataFrame()object
EXAMPLE:
TODO(new in release 1.5.0)
- apstools.utils.list_runs.summarize_runs(since=None, db=None)[source]
Report bluesky run metrics from the databroker.
How many different plans?
How many runs?
How many times each run was used?
How frequently? (TODO:)
PARAMETERS
- since
str : Report all runs since this ISO8601 date & time (default:
1995)- db
object : Instance of
databroker.Broker()(default:dbfrom the IPython shell)
Diagnostic Support for Memory
|
return memory used by this process |
Miscellaneous Support
|
convert text so it can be used as a dictionary key |
|
Given list of EPICS PV names, return dict of EpicsSignal objects. |
|
return a text table from |
|
Return the full dotted name |
|
Format a list of items. |
|
Show all the ophyd Signal and Device objects defined as globals. |
|
break a list (or other iterable) into pairs |
|
custom print the RunEngine metadata in a table |
|
print (stdout) a list of all snapshots in the databroker |
|
Set EPICS motor record's user coordinate to |
|
Replay the document stream from one (or more) scans (headers). |
|
(decorator) run |
|
make text safe to be used as an ophyd object name |
|
splits a line into words some of which might be quoted |
|
Encode |
|
|
String must not exceed EPICS PV length. |
|
|
Run a UNIX command, returns (stdout, stderr). |
- apstools.utils.misc.cleanupText(text)[source]
convert text so it can be used as a dictionary key
Given some input text string, return a clean version remove troublesome characters, perhaps other cleanup as well. This is best done with regular expression pattern matching.
- apstools.utils.misc.connect_pvlist(pvlist, wait=True, timeout=2, poll_interval=0.1)[source]
Given list of EPICS PV names, return dict of EpicsSignal objects.
PARAMETERS
- pvlist
[str] : list of EPICS PV names
- wait
bool : should wait for EpicsSignal objects to connect (default:
True)- timeout
float : maximum time to wait for PV connections, seconds (default: 2.0)
- poll_interval
float : time to sleep between checks for PV connections, seconds (default: 0.1)
- apstools.utils.misc.dictionary_table(dictionary, **kwargs)[source]
return a text table from
dictionaryPARAMETERS
- dictionary
dict : Python dictionary
Note: Keyword arguments parameters are kept for compatibility with previous versions of apstools. They are ignored now.
RETURNS
- table
object or
None:pyRestTable.Table()object (multiline text table) orNoneif dictionary has no contents
EXAMPLE:
In [8]: RE.md Out[8]: {'login_id': 'jemian:wow.aps.anl.gov', 'beamline_id': 'developer', 'proposal_id': None, 'pid': 19072, 'scan_id': 10, 'version': {'bluesky': '1.5.2', 'ophyd': '1.3.3', 'apstools': '1.1.5', 'epics': '3.3.3'}} In [9]: print(dictionary_table(RE.md)) =========== ============================================================================= key value =========== ============================================================================= beamline_id developer login_id jemian:wow.aps.anl.gov pid 19072 proposal_id None scan_id 10 version {'bluesky': '1.5.2', 'ophyd': '1.3.3', 'apstools': '1.1.5', 'epics': '3.3.3'} =========== =============================================================================
- apstools.utils.misc.full_dotted_name(obj)[source]
Return the full dotted name
The
.dotted_nameproperty does not include the name of the root object. This routine adds that.
- apstools.utils.misc.listobjects(show_pv=True, printing=True, verbose=False, symbols=None)[source]
Show all the ophyd Signal and Device objects defined as globals.
PARAMETERS
- show_pv
bool : If True, also show relevant EPICS PV, if available. (default: True)
- printing
bool : If True, print table to stdout. (default: True)
- verbose
bool : If True, also show
str(obj. (default: False)- symbols
dict : If None, use global symbol table. If not None, use provided dictionary. (default:
globals())
RETURNS
- object:
Instance of
pyRestTable.Table()
EXAMPLE:
In [1]: listobjects() ======== ================================ ============= name ophyd structure EPICS PV ======== ================================ ============= adsimdet MySingleTriggerSimDetector vm7SIM1: m1 EpicsMotor vm7:m1 m2 EpicsMotor vm7:m2 m3 EpicsMotor vm7:m3 m4 EpicsMotor vm7:m4 m5 EpicsMotor vm7:m5 m6 EpicsMotor vm7:m6 m7 EpicsMotor vm7:m7 m8 EpicsMotor vm7:m8 noisy EpicsSignalRO vm7:userCalc1 scaler ScalerCH vm7:scaler1 shutter SimulatedApsPssShutterWithStatus ======== ================================ ============= Out[1]: <pyRestTable.rest_table.Table at 0x7fa4398c7cf8> In [2]:
(new in apstools release 1.1.8)
- apstools.utils.misc.pairwise(iterable)[source]
break a list (or other iterable) into pairs
s -> (s0, s1), (s2, s3), (s4, s5), ... In [71]: for item in pairwise("a b c d e fg".split()): ...: print(item) ...: ('a', 'b') ('c', 'd') ('e', 'fg')
- apstools.utils.misc.print_RE_md(dictionary=None, fmt='simple', printing=True)[source]
custom print the RunEngine metadata in a table
PARAMETERS
- dictionary
dict : Python dictionary
EXAMPLE:
In [4]: print_RE_md() RunEngine metadata dictionary: ======================== =================================== key value ======================== =================================== EPICS_CA_MAX_ARRAY_BYTES 1280000 EPICS_HOST_ARCH linux-x86_64 beamline_id APS USAXS 9-ID-C login_id usaxs:usaxscontrol.xray.aps.anl.gov pid 67933 proposal_id testing Bluesky installation scan_id 0 versions ======== ===== key value ======== ===== apstools 1.1.3 bluesky 1.5.2 epics 3.3.1 ophyd 1.3.3 ======== ===== ======================== ===================================
- apstools.utils.misc.print_snapshot_list(db, printing=True, **search_criteria)[source]
print (stdout) a list of all snapshots in the databroker
USAGE:
print_snapshot_list(db, ) print_snapshot_list(db, purpose="this is an example") print_snapshot_list(db, since="2018-12-21", until="2019")
EXAMPLE:
In [16]: from apstools.utils import print_snapshot_list ...: from apstools.callbacks import SnapshotReport ...: print_snapshot_list(db, since="2018-12-21", until="2019") ...: = ======== ========================== ================== # uid date/time purpose = ======== ========================== ================== 0 d7831dae 2018-12-21 11:39:52.956904 this is an example 1 5049029d 2018-12-21 11:39:30.062463 this is an example 2 588e0149 2018-12-21 11:38:43.153055 this is an example = ======== ========================== ================== In [17]: SnapshotReport().print_report(db["5049029d"]) ======================================== snapshot: 2018-12-21 11:39:30.062463 ======================================== example: example 2 hints: {} iso8601: 2018-12-21 11:39:30.062463 look: can snapshot text and arrays too note: no commas in metadata plan_description: archive snapshot of ophyd Signals (usually EPICS PVs) plan_name: snapshot plan_type: generator purpose: this is an example scan_id: 1 software_versions: { 'python': '''3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]''', 'PyEpics': '3.3.1', 'bluesky': '1.4.1', 'ophyd': '1.3.0', 'databroker': '0.11.3', 'apstools': '0.0.38' } time: 1545413970.063167 uid: 5049029d-075c-453c-96d2-55431273852b ========================== ====== ================ =================== timestamp source name value ========================== ====== ================ =================== 2018-12-20 18:24:34.220028 PV compress [0.1, 0.2, 0.3] 2018-12-13 14:49:53.121188 PV gov:HOSTNAME otz.aps.anl.gov 2018-12-21 11:39:24.268148 PV gov:IOC_CPU_LOAD 0.22522317161410768 2018-12-21 11:39:24.268151 PV gov:SYS_CPU_LOAD 9.109026666525944 2018-12-21 11:39:30.017643 PV gov:iso8601 2018-12-21T11:39:30 2018-12-13 14:49:53.135016 PV otz:HOSTNAME otz.aps.anl.gov 2018-12-21 11:39:27.705304 PV otz:IOC_CPU_LOAD 0.1251210270549924 2018-12-21 11:39:27.705301 PV otz:SYS_CPU_LOAD 11.611234438304471 2018-12-21 11:39:30.030321 PV otz:iso8601 2018-12-21T11:39:30 ========================== ====== ================ =================== exit_status: success num_events: {'primary': 1} run_start: 5049029d-075c-453c-96d2-55431273852b time: 1545413970.102147 uid: 6c1b2100-1ef6-404d-943e-405da9ada882
- apstools.utils.misc.redefine_motor_position(motor, new_position)[source]
Set EPICS motor record’s user coordinate to
new_position.
- apstools.utils.misc.replay(headers, callback=None, sort=True)[source]
Replay the document stream from one (or more) scans (headers).
PARAMETERS
- headers
scan or [scan] : Scan(s) to be replayed through callback. A scan is an instance of a Bluesky
databroker.Header. see: https://nsls-ii.github.io/databroker/api.html?highlight=header#header-api- callback
scan or [scan] : The Bluesky callback to handle the stream of documents from a scan. If
None, then use the bec (BestEffortCallback) from the IPython shell. (default:None)- sort
bool : Sort the headers chronologically if True. (default:
True)
(new in apstools release 1.1.11)
- apstools.utils.misc.run_in_thread(func)[source]
(decorator) run
funcin threadUSAGE:
@run_in_thread def progress_reporting(): logger.debug("progress_reporting is starting") # ... #... progress_reporting() # runs in separate thread #...
- apstools.utils.misc.safe_ophyd_name(text)[source]
make text safe to be used as an ophyd object name
Given some input text string, return a clean version. Remove troublesome characters, perhaps other cleanup as well. This is best done with regular expression pattern matching.
The “sanitized” name fits this regular expression:
[A-Za-z_][\w_]*
Also can be used for safe HDF5 and NeXus names.
- apstools.utils.misc.split_quoted_line(line)[source]
splits a line into words some of which might be quoted
TESTS:
FlyScan 0 0 0 blank FlyScan 5 2 0 "empty container" FlyScan 5 12 0 "even longer name" SAXS 0 0 0 blank SAXS 0 0 0 "blank"
RESULTS:
['FlyScan', '0', '0', '0', 'blank'] ['FlyScan', '5', '2', '0', 'empty container'] ['FlyScan', '5', '12', '0', 'even longer name'] ['SAXS', '0', '0', '0', 'blank'] ['SAXS', '0', '0', '0', 'blank']
OverrideParameters
Define parameters that can be overridden from a user configuration file.
EXAMPLE:
Create an overrides object in a new file override_params.py:
import apstools.utils
overrides = apstools.utils.OverrideParameters()
When code supports a parameter for which a user can provide
a local override, the code should import the overrides
object (from the override_params module),
and then register the parameter name, such as this example:
from override_params import overrides
overrides.register("minimum_step")
Then later:
minstep = overrides.pick("minimum_step", 45e-6)
In the user’s configuration file that will override
the value of 45e-6 (such as can be loaded via
%run -i user.py), import the overrides`
object (from the override_params module):
from override_params import overrides
and then override the attribute(s) as desired:
overrides.set("minimum_step", 1.0e-5)
With this override in place, the minstep value
(from pick())
will be 1e-5.
Get a pandas DataFrame object with all the overrides:
overrides.summary()
which returns this table:
parameter value
0 minimum_step 0.00001
Define parameters that can be overridden from a user configuration file. |
- class apstools.utils.override_parameters.OverrideParameters[source]
Define parameters that can be overridden from a user configuration file.
NOTE: This is a pure Python object, not using ophyd.
pick(parameter, default)Return either the override parameter value if defined, or the default.
register(parameter_name)Register a new parameter name to be supported by user overrides.
reset(parameter_name)Remove an override value for a known parameter.
Remove override values for all known parameters.
set(parameter_name, value)Define an override value for a known parameter.
summary()Return a pandas DataFrame with all overrides.
(new in apstools 1.5.2)
- pick(parameter, default)[source]
Return either the override parameter value if defined, or the default.
Plot Support
|
Get the MatPlotLib Figure window for y vs x. |
|
Get the first live plot that matches |
|
Find the plot with axes x and y and replot with at most the last n lines. |
|
Find the plot(s) by name and replot with at most the last n lines. |
- apstools.utils.plot.select_live_plot(bec, signal)[source]
Get the first live plot that matches
signal.PARAMETERS
- bec
object: instance of
bluesky.callbacks.best_effort.BestEffortCallback- signal
object: The Y axis object (an
ophyd.Signal)
RETURNS
- object:
Instance of
bluesky.callbacks.best_effort.LivePlotPlusPeaks()orNone
- apstools.utils.plot.select_mpl_figure(x, y)[source]
Get the MatPlotLib Figure window for y vs x.
PARAMETERS
- x
object: X axis object (an
ophyd.Signal)- y
ophyd object: X axis object (an
ophyd.Signal)
RETURNS
- object or
None: Instance of
matplotlib.pyplot.Figure()
- apstools.utils.plot.trim_plot_by_name(n=3, plots=None)[source]
Find the plot(s) by name and replot with at most the last n lines.
Note: this is not a bluesky plan. Call it as normal Python function.
It is recommended to call
trim_plot_by_name()before the scan(s) that generate plots. Plots are generated from a RunEngine callback, executed after the scan completes.PARAMETERS
- n
int : number of plots to keep
- plots
str, [str], or None : name(s) of plot windows to trim (default: all plot windows)
EXAMPLES:
trim_plot_by_name() # default of n=3, apply to all plots trim_plot_by_name(5) # change from default of n=3 trim_plot_by_name(5, "noisy_det vs motor") # just this plot trim_plot_by_name( 5, ["noisy_det vs motor", "det noisy_det vs motor"]] )
EXAMPLE:
# use simulators from ophyd from bluesky import plans as bp from bluesky import plan_stubs as bps from ophyd.sim import * snooze = 0.25 def scan_set(): trim_plot_by_name() yield from bp.scan([noisy_det], motor, -1, 1, 5) yield from bp.scan([noisy_det, det], motor, -2, 1, motor2, 3, 1, 6) yield from bps.sleep(snooze) # repeat the_scans 15 times uids = RE(bps.repeat(scan_set, 15))
(new in release 1.3.5)
- apstools.utils.plot.trim_plot_lines(bec, n, x, y)[source]
Find the plot with axes x and y and replot with at most the last n lines.
Note:
trim_plot_lines()is not a bluesky plan. Call it as normal Python function.EXAMPLE:
trim_plot_lines(bec, 1, m1, noisy)
PARAMETERS
- bec
object : instance of BestEffortCallback
- n
int : number of plots to keep
- x
object : instance of ophyd.Signal (or subclass), independent (x) axis
- y
object : instance of ophyd.Signal (or subclass), dependent (y) axis
(new in release 1.3.5)
Support for IPython profiles
|
get the IPython shell's namespace dictionary (or globals() if not found) |
return the name of the current ipython profile or |
|
get the IPython shell's namespace dictionary (or empty if not found) |
- apstools.utils.profile_support.getDefaultNamespace(attr='user_ns')[source]
get the IPython shell’s namespace dictionary (or globals() if not found)
EPICS PV Registry
|
Find the ophyd (dotted name) object associated with the given ophyd name. |
|
Find all ophyd objects associated with the given EPICS PV. |
|
Cross-reference EPICS PVs with ophyd EpicsSignalBase objects. |
- class apstools.utils.pvregistry.PVRegistry(ns=None)[source]
Cross-reference EPICS PVs with ophyd EpicsSignalBase objects.
- apstools.utils.pvregistry.findbyname(oname, force_rebuild=False, ns=None)[source]
Find the ophyd (dotted name) object associated with the given ophyd name.
PARAMETERS
- oname
str : ophyd name to search
- force_rebuild
bool : If
True, rebuild the internal registry that maps ophyd names to ophyd objects.- ns
dict or None : Namespace dictionary of Python objects.
RETURNS
- str or
None: Name of the ophyd object.
EXAMPLE:
In [45]: findbyname("adsimdet_cam_acquire") Out[45]: 'adsimdet.cam.acquire'
(new in apstools 1.5.0)
- apstools.utils.pvregistry.findbypv(pvname, force_rebuild=False, ns=None)[source]
Find all ophyd objects associated with the given EPICS PV.
PARAMETERS
- pvname
str : EPICS PV name to search
- force_rebuild
bool : If
True, rebuild the internal registry that maps EPICS PV names to ophyd objects.- ns
dict or None : Namespace dictionary of Python objects.
RETURNS
- dict or
None: Dictionary of matching ophyd objects, keyed by how the PV is used by the ophyd signal. The keys are
readandwrite.
EXAMPLE:
In [45]: findbypv("ad:cam1:Acquire") Out[45]: {'read': [], 'write': ['adsimdet.cam.acquire']} In [46]: findbypv("ad:cam1:Acquire_RBV") Out[46]: {'read': ['adsimdet.cam.acquire'], 'write': []}
Searching databroker catalogs
|
Searches the databroker v2 database. |
- apstools.utils.query.db_query(db, query)[source]
Searches the databroker v2 database.
PARAMETERS
- db
object : Bluesky database, an instance of
databroker.catalog.- query
dict : Search parameters.
RETURNS
- object :
Bluesky database, an instance of
databroker.catalogsatisfying thequeryparameters.
See also
databroker.catalog.search()
Common support of slits
|
Slit size and center as a named tuple |
- class apstools.utils.slit_core.SlitGeometry(width, height, x, y)
Slit size and center as a named tuple
- height
Alias for field number 1
- width
Alias for field number 0
- x
Alias for field number 2
- y
Alias for field number 3
Spreadsheet Support
|
base class: read-only support for Excel files, treat them like databases |
|
Generic (read-only) handling of Excel spreadsheet-as-database |
|
Exception when reading Excel spreadsheet. |
- class apstools.utils.spreadsheet.ExcelDatabaseFileBase(ignore_extra=True)[source]
base class: read-only support for Excel files, treat them like databases
Use this class when creating new, specific spreadsheet support.
EXAMPLE
Show how to read an Excel file where one of the columns contains a unique key. This allows for random access to each row of data by use of the key.
class ExhibitorsDB(ExcelDatabaseFileBase): ''' content for exhibitors from the Excel file ''' EXCEL_FILE = os.path.join("resources", "exhibitors.xlsx") LABELS_ROW = 2 def handle_single_entry(self, entry): '''any special handling for a row from the Excel file''' pass def handleExcelRowEntry(self, entry): '''identify unique key (row of the Excel file)''' key = entry["Name"] self.db[key] = entry
- class apstools.utils.spreadsheet.ExcelDatabaseFileGeneric(filename, labels_row=3, ignore_extra=True)[source]
Generic (read-only) handling of Excel spreadsheet-as-database
Note
This is the class to use when reading Excel spreadsheets.
In the spreadsheet, the first sheet should contain the table to be used. By default (see keyword parameter
labels_row), the table should start in cell A4. The column labels are given in row 4. A blank column should appear to the right of the table (see keyword parameterignore_extra). The column labels will describe the action and its parameters. Additional columns may be added for metadata or other purposes.The rows below the column labels should contain actions and parameters for those actions, one action per row.
To make a comment, place a
#in the action column. A comment should be ignored by the bluesky plan that reads this table. The table will end with a row of empty cells.While it’s a good idea to put the
actioncolumn first, that is not necessary. It is not even necessary to name the columnaction. You can re-arrange the order of the columns and change their names as long as the column names match what text strings your Python code expects to find.A future upgrade 1 will allow the table boundaries to be named by Excel when using Excel’s
Format as Table2 feature. For now, leave a blank row and column at the bottom and right edges of the table.- 1
- 2
Excel’s
Format as Table: https://support.office.com/en-us/article/Format-an-Excel-table-6789619F-C889-495C-99C2-2F971C0E2370
PARAMETERS
- filename
str : name (absolute or relative) of Excel spreadsheet file
- labels_row
int : Row (zero-based numbering) of Excel file with column labels, default:
3(Excel row 4)- ignore_extra
bool : When
True, ignore any cells outside of the table, default:True.Note that when
True, a row of cells within the table will be recognized as the end of the table, even if there are actions in following rows. To force an empty row, use a comment symbol#(actually, any non-empty content will work).When
False, cells with other information (in Sheet 1) will be made available, sometimes with unpredictable results.
EXAMPLE
See section Example: the run_command_file() plan for more examples.
(See also example screen shot.) Table (on Sheet 1) begins on row 4 in first column:
1 | some text here, maybe a title 2 | (could have content here) 3 | (or even more content here) 4 | action | sx | sy | sample | comments | | <-- leave empty column 5 | close | | | close the shutter | | 6 | image | 0 | 0 | dark | dark image | | 7 | open | | | | open the shutter | | 8 | image | 0 | 0 | flat | flat field image | | 9 | image | 5.1 | -3.2 | 4140 steel | heat 9172634 | | 10 | scan | 5.1 | -3.2 | 4140 steel | heat 9172634 | | 11 | scan | 0 | 0 | blank | | | 12 | 13 | ^^^ leave empty row ^^^ 14 | (could have content here)
Example python code to read this spreadsheet:
from apstools.utils import ExcelDatabaseFileGeneric, cleanupText def myExcelPlan(xl_file, md={}): excel_file = os.path.abspath(xl_file) xl = ExcelDatabaseFileGeneric(excel_file) for i, row in xl.db.values(): # prepare the metadata _md = {cleanupText(k): v for k, v in row.items()} _md["xl_file"] = xl_file _md["excel_row_number"] = i+1 _md.update(md) # overlay with user-supplied metadata # determine what action to take action = row["action"].lower() if action == "open": yield from bps.mv(shutter, "open") elif action == "close": yield from bps.mv(shutter, "close") elif action == "image": # your code to take an image, given **row as parameters yield from my_image(**row, md=_md) elif action == "scan": # your code to make a scan, given **row as parameters yield from my_scan(**row, md=_md) else: print(f"no handling for row {i+1}: action={action}") # execute this plan through the RunEngine RE(myExcelPlan("spreadsheet.xlsx", md=dict(purpose="apstools demo"))