Advanced Tutorial ================= .. currentmodule:: pyxnat This advanced tutorial is not much more complicated than the one for beginners. It only reviews parts of the API that may be less used (not that they are less useful!) and that are more likely to change in future releases. Introspection ------------- In order to browse a database people have to be aware of: - the REST hierarchy - schema types and fields - values of fields and resources within a project The idea of this interface is to help users find their way around a XNAT server by making it easier to gather the preceding information. Searchable datatypes and fields ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> # simple datatypes listing >>> central.inspect.datatypes() [..., 'xnat:subjectData', 'xnat:projectData', 'xnat:mrSessionData', ...] >>> # datatypes listing with filter >>> central.inspect.datatypes('cnda:*') ['cnda:manualVolumetryData', 'cnda:clinicalAssessmentData', 'cnda:psychometricsData', 'cnda:dtiData', 'cnda:atlasScalingFactorData', 'cnda:segmentationFastData', 'cnda:modifiedScheltensData'] >>> # simple fields listing >>> central.inspect.datatypes('xnat:subjectData') ['xnat:subjectData/SUBJECT_ID', 'xnat:subjectData/INSERT_DATE', 'xnat:subjectData/INSERT_USER', 'xnat:subjectData/GENDER_TEXT', ...] >>> # field listing with filter >>> central.inspect.datatypes('xnat:subjectData', '*ID*') ['xnat:subjectData/SUBJECT_ID', 'xnat:subjectData/ADD_IDS'] >>> # field listing on multiple types >>> central.inspect.datatypes('cnda:*', 'EXPT_ID') ['cnda:manualVolumetryData/EXPT_ID', 'cnda:clinicalAssessmentData/EXPT_ID', 'cnda:psychometricsData/EXPT_ID', 'cnda:dtiData/EXPT_ID', 'cnda:atlasScalingFactorData/EXPT_ID', 'cnda:segmentationFastData/EXPT_ID', 'cnda:modifiedScheltensData/EXPT_ID'] To known what values fields can take in the database:: >>> central.inspect.field_values('xnat:mrSessionData/SESSION_ID') REST hierarchy ~~~~~~~~~~~~~~ :mod:`pyxnat` does not support all the REST resources. The reasons for this is that, some of these resources are still experimental, or do not work exactly the same way which would make it difficult to provide a consistent interface at the Python level. However support for these exotic resources will increase in future releases. A good way to know what is the supported REST hierarchy is to use the following method:: >>> central.inspect.structure() - PROJECTS + SUBJECTS + EXPERIMENTS + ASSESSORS + RESOURCES + FILES + IN_RESOURCES + FILES + OUT_RESOURCES + FILES + RECONSTRUCTIONS + IN_RESOURCES + FILES + OUT_RESOURCES + FILES + SCANS + RESOURCES + FILES + RESOURCES + FILES + RESOURCES + FILES + RESOURCES + FILES Naming conventions ~~~~~~~~~~~~~~~~~~ Administrators usually use a consistent vocabulary across single projects, that maps to XNAT datatypes. A new feature in introduced in 0.6 and improved in 0.7 is to be able to define a mapping so that specific name patterns can be used to cast a resource when creating a new one. For example with the following mapping:: '/projects/my_project/subjects/*/experiments/SessionA_*':'xnat:mrSessionData' Creating an experiment in ``my_project`` that matches `Session_*`, creates an `xnat:mrSessionData`:: >>> central.select('/projects/my_project/subjects/*/experiments/SessionA_new').create() In the 0.7, it is no longer up to the user to manually save and load the mapping file. Files are created automatically and the mappings are discovered `on the fly` when queries are issued on the server. Files are loaded at the ``Interface`` creation and the mappings are updated regularly. This functionality can be configured with the following method:: >>> # activate (default) >>> central.inspect.set_autolearn('True') >>> # setup update frequency >>> central.inspect.set_autolearn(tick=10) When a mapping is available, re-running the ``rest_hierarchy`` method will display additional information such as:: - PROJECTS + SUBJECTS + EXPERIMENTS ----------- - xnat:mrSessionData - xnat:petSessionData +ASSESSORS .... There are additional methods to visualize and display the mappings:: >>> central.inspect.experiment_types() >>> central.inspect.assessor_types() >>> central.inspect.scan_types() >>> central.inspect.reconstruction_types() Methods also allow to have a quick look on the values at those levels on the database:: >>> central.inspect.experiment_values('xnat:mrSessionData') >>> central.inspect.assessor_values('xnat:mrSessionData') >>> central.inspect.scan_values('xnat:mrSessionData') >>> central.inspect.reconstruction_values('xnat:mrSessionData') For more details check the reference documentation. .. note:: With ``networkx`` and ``matplotlib`` installed, a ``draw`` subinterface will be made available to display some data from the inspect subinterface as a graph:: >>> central.draw.experiments() >>> central.draw.assessors() >>> central.draw.scans() >>> central.draw.reconstructions() >>> central.draw.architecture() >>> central.draw.field_values() Sharing ------- It is possible to share ``Subjects``, ``Experiments`` and ``Assessors`` via the REST API. The methods to control sharing are:: >>> subject = interface.select('/project/project1/subject/subject1') >>> subject.share('project2') >>> subject.unshare('project2') >>> # to know to in which projects a subject is available >>> subject.shares() Almost the same interface is available for collection objects:: >>> subjects = interface.select('/project/project1/subjects') >>> subjects.share('project2') >>> subjects.unshare('project2') >>> # to retrieve the subjects sharing a list of projects >>> subjects.sharing(['project1', 'project2']) .. note:: Of course the permissions policies (user level and project accessibility)still apply. .. warning:: The ``shares`` and ``sharing`` methods are not implemented in an efficient way at the moment. There is another more concerning issue: subjects for example are accessible through their ID or label. But labels stop working when trying to access a subject through a project that is not its orginial one. Search templates ---------------- :mod:`pyxnat` is also able to define templates to use with XNAT search engine. They work basically the same way as usual searches but instead of defining values to filter the data, one need to define keywords to replace them later with the actual values:: >>> contraints = [('xnat:subjectData/SUBJECT_ID','LIKE','subject_id'), ('xnat:subjectData/PROJECT', '=', 'project_id'), 'OR', [('xnat:subjectData/AGE','>','age'), 'AND' ] ] >>> columns = ['xnat:subjectData/PROJECT', 'xnat:subjectData/SUBJECT_ID'] >>> interface.manage.search.save_template('name', 'xnat:subjectData', columns, criteria, sharing='public', description='my first template' ) >>> interface.manage.search.use_template('name', {'subject_id':'%', 'project_id':'my_project', 'age':'42' } ) >>> interface.select(...).where(template=('name', {'subject_id':'%', 'project_id':'my_project', 'age':'42'} ) ) And now it is also possible to reuse saved searches in the where clause in the same way as the templates. It means that you reuse the constraints but not the data selection which still changes: >>> interface.select(...).where(query='saved_name') Provenance definition --------------------- :mod:`pyxnat` 0.8 introduces a way to store provenance i.e. to describe the steps that were performed on an initial data to produce this one. Reconstructions and assessors only can be annotated with provenance information: >>> prov = {'program':'young', 'timestamp':'2011-03-01T12:01:01.897987', 'user':'angus', 'machine':'war', 'platform':'linux', } >>> element.provenance.attach(prov) >>> element.provenance.get() >>> element.dettach() The provenance :meth:`Provenance.attach` method adds new steps with each call, unless the `overwrite` parameter is set to `True`. The following keywords for the provenance dictionary are available: - program - program_version - program_arguments - timestamp - cvs - user - machine - platform - platform_version - compiler - compiler_version