Difference between revisions of "Technical documentation of management systems, data processing and publication tools not specialised on collection data"

From GFBio Public Wiki
Jump to: navigation, search
 
(38 intermediate revisions by 9 users not shown)
Line 5: Line 5:
 
The tools and systems below are set up for management, processing and publishing of data from more than one data domain: collection and observation data, trait data, taxonomic, taxon reference and checklist data, sequence data, sampling data, survey data as well as scientific data packages as a whole. Some are browser-based applications with own online portal, others are usable as desk top stand-alone applications or are part of a client-server network. All are involved in the GFBio dataflow and part of the portfolios of GFBio data centers.  
 
The tools and systems below are set up for management, processing and publishing of data from more than one data domain: collection and observation data, trait data, taxonomic, taxon reference and checklist data, sequence data, sampling data, survey data as well as scientific data packages as a whole. Some are browser-based applications with own online portal, others are usable as desk top stand-alone applications or are part of a client-server network. All are involved in the GFBio dataflow and part of the portfolios of GFBio data centers.  
  
−
In parallel, the GFBio collection data centers documented their installations of [[Technical documentation of collection management systems at the GFBio Collection Data Centers|collection management systems]], [[Technical documentation of multimedia data management systems|multimedia data management systems]] and of [[Technical documentation of long-term archiving solutions at the GFBio collection data centers|long-term archiving solutions]].
+
In parallel, the GFBio collection data centers documented their installations of [[Technical documentation of collection management systems at the GFBio Collection Data Centers|collection management systems]], [[Technical documentation of multimedia data management systems|multimedia data management systems]], [[Technical documentation of long-term archiving solutions at the GFBio collection data centers|long-term archiving solutions]] and of [[Technical documentation of GFBio related IT services, tools and databases at the data centers|GFBio related IT services, tools and databases at the data centers]].
  
 
{| style="margin:1em 1em 1em 0; background:#FFFFFF;border:2px #000000 solid; border-collapse:collapse; empty-cells:show;" cellpadding="5" border="1"
 
{| style="margin:1em 1em 1em 0; background:#FFFFFF;border:2px #000000 solid; border-collapse:collapse; empty-cells:show;" cellpadding="5" border="1"
 
! style="width:5%" |
 
! style="width:5%" |
 
! style="width:12%" | Database system, Homepage
 
! style="width:12%" | Database system, Homepage
−
! style="width:12%" | AQUiLA
+
! style="width:12%" | [https://search.senckenberg.de/aquila-public-search/search AQUiLA]
 
! style="width:12%" | [http://bacdive.dsmz.de BacDive]
 
! style="width:12%" | [http://bacdive.dsmz.de BacDive]
−
! style="width:12%" | BioCASe Monitor Service
+
! style="width:12%" | BioCASe Monitor Service (GFBio version)
 
! style="width:12%" |[http://www.biocase.org/products/provider_software/index.shtml BioCASe Provider Software]
 
! style="width:12%" |[http://www.biocase.org/products/provider_software/index.shtml BioCASe Provider Software]
 
! style="width:12%" | [http://diversityworkbench.net/Portal/DiversityDescriptions DiversityDescriptions] (DD)
 
! style="width:12%" | [http://diversityworkbench.net/Portal/DiversityDescriptions DiversityDescriptions] (DD)
Line 18: Line 18:
 
! style="width:12%" | [http://diversityworkbench.net/Portal/DiversityTaxonNames DiversityTaxonNames] (DTN)
 
! style="width:12%" | [http://diversityworkbench.net/Portal/DiversityTaxonNames DiversityTaxonNames] (DTN)
 
! style="width:12%" | [http://cybertaxonomy.eu/taxeditor/ EDIT Taxonomic Editor]
 
! style="width:12%" | [http://cybertaxonomy.eu/taxeditor/ EDIT Taxonomic Editor]
−
! style="width:12%" | [https://knb.ecoinformatics.org/knb/docs/index.html Metacat]
 
 
! style="width:12%" | [https://www.morphdbase.de/ Morph-D-Base]
 
! style="width:12%" | [https://www.morphdbase.de/ Morph-D-Base]
 
! style="width:12%" | [http://rebind.bgbm.org/ reBiND]
 
! style="width:12%" | [http://rebind.bgbm.org/ reBiND]
Line 32: Line 31:
 
|SMNS, SNSB, ZFMK
 
|SMNS, SNSB, ZFMK
 
|BGBM
 
|BGBM
−
|SGN
 
 
|ZFMK
 
|ZFMK
 
|BGBM
 
|BGBM
 
|-  
 
|-  
 
|'''Contact persons'''
 
|'''Contact persons'''
−
|Alexander Schmid and A. Allspach (SGN)
+
|T. Winter, A. Penzlin and A. Allspach (SGN)
−
|C. Söhngen, A. Podstawka and B. Bunk (Mailcontact of BacDive Team: mailto:contact@bacdive.de)  
+
|L.Reimer (Mailcontact of BacDive Team: mailto:contact@bacdive.de)  
−
|F. Glöckler, J. Hoffmann (MfN)
+
|F. Glöckler (MfN)
 
|J. Holetschek (BGBM)
 
|J. Holetschek (BGBM)
−
|V. Sanz and A. Link (SNSB)
+
|A. Link (SNSB)
 
|J.C. Monje (SMNS), T. Weibulat and D. Triebel (SNSB)
 
|J.C. Monje (SMNS), T. Weibulat and D. Triebel (SNSB)
 
|J.C. Monje (SMNS), D. Triebel (SNSB) and P. Grobe (ZFMK)  
 
|J.C. Monje (SMNS), D. Triebel (SNSB) and P. Grobe (ZFMK)  
 
|A. Müller and K. Luther (BGBM)
 
|A. Müller and K. Luther (BGBM)
−
| E.-M. Gerstner and C. Weiland (SGN)
 
 
| L. Vogt (Uni Bonn) and P. Grobe (ZFMK)
 
| L. Vogt (Uni Bonn) and P. Grobe (ZFMK)
 
| D. Fichtmüller (BGBM)
 
| D. Fichtmüller (BGBM)
 
|-  
 
|-  
 
|'''Developer group, country'''
 
|'''Developer group, country'''
−
|Senckenberg IT Services; Team Application Development
+
|Biodiversity Informatics SGN
 
|DSMZ IT Services & Bioinformatics Group
 
|DSMZ IT Services & Bioinformatics Group
 
|Department of Science Data Management at the MfN Berlin, Germany  
 
|Department of Science Data Management at the MfN Berlin, Germany  
Line 58: Line 55:
 
|SNSB IT Center Germany (together with MfN and UBT as far as the whole DWB platform is concerned)
 
|SNSB IT Center Germany (together with MfN and UBT as far as the whole DWB platform is concerned)
 
|BGBM (and Naturalis, NL implementing the Validation Framework)
 
|BGBM (and Naturalis, NL implementing the Validation Framework)
−
| Knowledge Network for Biocomplexity ([http://knb.ecoinformatics.org/ KNB]), U.S.A.
 
 
| Biodiversity Informatics ZFMK
 
| Biodiversity Informatics ZFMK
 
| Biodiversity Informatics BGBM, Germany
 
| Biodiversity Informatics BGBM, Germany
Line 65: Line 61:
 
|collection data, collection management data, observation data
 
|collection data, collection management data, observation data
 
|scientific collection data, trait data, physiological, morphological and environmental descriptions
 
|scientific collection data, trait data, physiological, morphological and environmental descriptions
−
|
+
|data mapped in BioCASe Provider Software
 
|collection data, collection management data, observation data  
 
|collection data, collection management data, observation data  
 
|triple-structured scientific data, e. g. any kind of descriptive or trait data, attribute data, molecular sequence data, sampling data; matrix data import; interactive key functions
 
|triple-structured scientific data, e. g. any kind of descriptive or trait data, attribute data, molecular sequence data, sampling data; matrix data import; interactive key functions
Line 71: Line 67:
 
|taxonomic data, taxon reference and checklist data
 
|taxonomic data, taxon reference and checklist data
 
|collection data, taxonomic data (core functionality), molecular data, descriptive data, identification keys
 
|collection data, taxonomic data (core functionality), molecular data, descriptive data, identification keys
−
| scientific data packages, data sets and data tables, particularly from ecology and environmental science
 
 
| morphological descriptions, taxonomic data, character matrices; current project: ontology based descriptions
 
| morphological descriptions, taxonomic data, character matrices; current project: ontology based descriptions
 
|collection data, collection management data, observation data  
 
|collection data, collection management data, observation data  
Line 78: Line 73:
 
|all recent and fossil organisms groups; geological data
 
|all recent and fossil organisms groups; geological data
 
|Bacteria and Archaea (other groups of microorganisms may be in future releases)  
 
|Bacteria and Archaea (other groups of microorganisms may be in future releases)  
−
|
+
|not applicable
 
|all recent and fossil organism groups  
 
|all recent and fossil organism groups  
 
|all
 
|all
Line 84: Line 79:
 
|all recent and fossil organism groups
 
|all recent and fossil organism groups
 
|all recent and fossil organism groups (potentially)
 
|all recent and fossil organism groups (potentially)
−
| all
 
 
| all
 
| all
 
| all recent and fossil organism groups  
 
| all recent and fossil organism groups  
Line 90: Line 84:
 
| rowspan="17" | Software and Database System  
 
| rowspan="17" | Software and Database System  
 
|'''Server Operating System'''
 
|'''Server Operating System'''
−
|Ubuntu, Apache Tomcat, Mapserver
+
|Ubuntu, Apache Tomcat
 
| Ubuntu 12.04 (Apache)
 
| Ubuntu 12.04 (Apache)
 
|currently on Linux, generally supported by any OS running Apache, SQLite and PHP
 
|currently on Linux, generally supported by any OS running Apache, SQLite and PHP
−
|MS Windows, Linux, Unix, Mac (any OS that runs Python 2.5, 2.6, 2.7)
+
|MS Windows, Linux, Unix, Mac (any OS that runs Python 3); Also available as Docker image.
−
|MS Windows Server 2008 R2, 2012 R2; (clients from MS Windows XP to MS Windows 10)
+
|MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10)
−
|MS Windows Server 2008 R2, 2012 R2; (clients from MS Windows XP to MS Windows 10)
+
|MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10)
−
|MS Windows Server 2008 R2, 2012 R2; (clients from MS Windows XP to MS Windows 10)
+
|MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10)
 
|Linux, Mac OS, MS Windows
 
|Linux, Mac OS, MS Windows
−
| Linux, Mac OS, MS Windows
 
 
| Linux version 2.6.
 
| Linux version 2.6.
 
| currently on Linux, generally supported by Mac OS, MS Windows as well
 
| currently on Linux, generally supported by Mac OS, MS Windows as well
Line 106: Line 99:
 
|MySQL 5.1
 
|MySQL 5.1
 
|SQLite
 
|SQLite
−
|MS SQL Server, MySQL, Access, Oracle, Postgres, 4D, Firebirs, Foxpro, Sybase; Excel, CSV
+
|MS SQL Server, MySQL, Access, Sqlite, Oracle, Postgres, Firebird, Sybase; Excel, CSV
−
|MS SQL-Server 2014
+
|MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express
−
|MS SQL-Server 2014
+
|MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express
−
|MS SQL-Server 2014
+
|MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express
 
|MySQL, PostgreSQL, H2, SQL Server
 
|MySQL, PostgreSQL, H2, SQL Server
−
| PostgreSQL (or another SQL92-compliant RDBMS like Oracle)
 
 
| MySQL, ZOPE ODB
 
| MySQL, ZOPE ODB
 
| eXist-db
 
| eXist-db
Line 124: Line 116:
 
|MS SQL-Server specific; few DWB specific features and roles added
 
|MS SQL-Server specific; few DWB specific features and roles added
 
|Roles can be assigned to users according to taxonomic groups and data types (e.g. distribution data only). Project admins have rights to edit all types of data. The underlying software library allows very granular user rights assignment (not yet implemented in the user interface)
 
|Roles can be assigned to users according to taxonomic groups and data types (e.g. distribution data only). Project admins have rights to edit all types of data. The underlying software library allows very granular user rights assignment (not yet implemented in the user interface)
−
| Metacat supports internal password file authentication or the use of LDAP as an external authentication mechanism; multiple access right control (specification of rwx rights to single persons, user groups, etc.) in combination with [https://knb.ecoinformatics.org/#tools/morpho Morpho], a Java-based import wizard for EML which interfaces with the KNB Metacat server
 
 
|Group based user rights. Entries can be shared read-only or writable within groups. Released entries readable by the public  
 
|Group based user rights. Entries can be shared read-only or writable within groups. Released entries readable by the public  
 
| Group based user rights managment through the eXist-db
 
| Group based user rights managment through the eXist-db
Line 137: Line 128:
 
|C# desktop application (rich client)  
 
|C# desktop application (rich client)  
 
|desktop application (rich-client), browser based client (for less complex operations, in development)
 
|desktop application (rich-client), browser based client (for less complex operations, in development)
−
|Browser based client
 
 
|Browser based client
 
|Browser based client
 
|any modern browser
 
|any modern browser
Line 144: Line 134:
 
|PostGIS extension, Mapserver (WMS/WFS)
 
|PostGIS extension, Mapserver (WMS/WFS)
 
| -
 
| -
−
| -
+
| not applicable
 
| not applicable
 
| not applicable
 
| -
 
| -
Line 150: Line 140:
 
| -
 
| -
 
|Visualization of distribution and point maps
 
|Visualization of distribution and point maps
−
| PostGIS extension, Geoserver (WMS/WFS)
 
 
| None
 
| None
 
| None
 
| None
 
|-
 
|-
 
|'''GUIs for data import'''
 
|'''GUIs for data import'''
−
|generic import tool; import wizard coming soon
+
|generic import tool
 
| -
 
| -
−
| interactive handling of data
+
| web forms for metadata entry, metadata retrieval from BioCASe Provider Software
 
| -  
 
| -  
 
|Import wizard (DELTA, SDD, CSV), matrix wizard, local import (FASTA, FASTQ)  
 
|Import wizard (DELTA, SDD, CSV), matrix wizard, local import (FASTA, FASTQ)  
−
|generic import tool (coming soon); CSV
+
|generic import tool; CSV
 
|generic import tool, import wizard; CSV  
 
|generic import tool, import wizard; CSV  
 
| Currently very simple (but more complex UIs planned)
 
| Currently very simple (but more complex UIs planned)
−
| Webform or Morpho
 
 
| direct into the database, bulk upload of images possible
 
| direct into the database, bulk upload of images possible
 
| Admin interface of eXist-db allows for GUI based upload of files and directory managment
 
| Admin interface of eXist-db allows for GUI based upload of files and directory managment
Line 170: Line 158:
 
|EXCEL, CSV
 
|EXCEL, CSV
 
|CSV,PDF, via webservice: XML, JSON
 
|CSV,PDF, via webservice: XML, JSON
−
| Webservices (JSON)
+
| GUI for API usage (JSON)
 
| ABCD and DwC archive export
 
| ABCD and DwC archive export
−
|Export (DELTA, SDD, CSV, FASTA, FASTQ)   
+
|Export (DELTA, SDD, EML, CSV, FASTA, FASTQ)   
 
|various export tools, export wizard (coming soon)
 
|various export tools, export wizard (coming soon)
 
|various export tools (e. g. for hierarchy, for accepted names etc.), export wizard; CSV
 
|various export tools (e. g. for hierarchy, for accepted names etc.), export wizard; CSV
 
|Currently very simple (but more complex UIs planned)
 
|Currently very simple (but more complex UIs planned)
−
| Export format: EML, Download via Webform or Morpho
 
 
| CSV, RDF (in development)
 
| CSV, RDF (in development)
 
| None
 
| None
Line 189: Line 176:
 
|default: english (multilingual through translation tables)
 
|default: english (multilingual through translation tables)
 
|English and German (Easy to extend for other languages)
 
|English and German (Easy to extend for other languages)
−
| English
 
 
| English
 
| English
 
| English
 
| English
Line 196: Line 182:
 
| -
 
| -
 
| -
 
| -
−
| Yes, [http://bms.gfbio.org]
+
|Yes
 
|not applicable
 
|not applicable
 
|Yes, [http://diversityworkbench.net/Portal/DiversityDescriptions DD software download]
 
|Yes, [http://diversityworkbench.net/Portal/DiversityDescriptions DD software download]
Line 202: Line 188:
 
|Yes, [http://diversityworkbench.net/Portal/DiversityTaxonNames DTN software download]
 
|Yes, [http://diversityworkbench.net/Portal/DiversityTaxonNames DTN software download]
 
|Yes
 
|Yes
−
|Yes
 
 
|Yes
 
|Yes
 
|Yes
 
|Yes
Line 209: Line 194:
 
| -
 
| -
 
| -
 
| -
−
|Yes, [https://github.com/MfN-Berlin/BioCASe-Monitor-Service]
+
|Yes, [https://github.com/MfN-Berlin/BioCASe-Monitor-Service MfN GitHub]
 
|Yes, [http://ww2.biocase.org/svn/bps2/branches/stable/ BPS SVN code repository]
 
|Yes, [http://ww2.biocase.org/svn/bps2/branches/stable/ BPS SVN code repository]
 
|Yes, [http://svn.snsb.info/repos/DiversityWorkbench/ DWB SVN code repository]
 
|Yes, [http://svn.snsb.info/repos/DiversityWorkbench/ DWB SVN code repository]
Line 215: Line 200:
 
|Yes, [http://svn.snsb.info/repos/DiversityWorkbench/ DWB SVN code repository]
 
|Yes, [http://svn.snsb.info/repos/DiversityWorkbench/ DWB SVN code repository]
 
|Yes, http://cybertaxonomy.eu/taxeditor/source-repository.html
 
|Yes, http://cybertaxonomy.eu/taxeditor/source-repository.html
−
| Yes, (Download [https://knb.ecoinformatics.org/knb/docs/ source distribution])
 
 
| Available on request
 
| Available on request
 
| Yes, [http://wiki.bgbm.org/rebind_documentation/index.php/Installation Details about reBiND Installation, Prerequisites and SVN code repository]  
 
| Yes, [http://wiki.bgbm.org/rebind_documentation/index.php/Installation Details about reBiND Installation, Prerequisites and SVN code repository]  
Line 228: Line 212:
 
|GPL v.2
 
|GPL v.2
 
|Mozilla Public License Version 1.2 (Project License)
 
|Mozilla Public License Version 1.2 (Project License)
−
| [https://knb.ecoinformatics.org/knb/docs/license.html GPL]
 
 
| GPL
 
| GPL
 
| Mozilla Public License (MPL) 2.0
 
| Mozilla Public License (MPL) 2.0
Line 241: Line 224:
 
|[http://diversityworkbench.net/Portal/DiversityTaxonNames_Information_Models DTN data model]
 
|[http://diversityworkbench.net/Portal/DiversityTaxonNames_Information_Models DTN data model]
 
|Common Data Model (CDM) http://cybertaxonomy.eu/cdm/latest/
 
|Common Data Model (CDM) http://cybertaxonomy.eu/cdm/latest/
−
| -
 
 
| Coming soon
 
| Coming soon
 
| No
 
| No
 
|-
 
|-
 
|'''State of development'''
 
|'''State of development'''
−
| ongoing; beta version (search portal) online: https://search.senckenberg.de/aquila-public-search/search, http://www.senckenberg.de/root/index.php?page_id=2868
+
|ongoing; beta version (search portal) online: https://search.senckenberg.de/aquila-public-search/search, https://www.senckenberg.de/en/science/research-infrastructure/collections/digitale-ressourcen/aquila/  
 
|ongoing
 
|ongoing
−
|ongoing (latest release 2.0, https://github.com/MfN-Berlin/BioCASe-Monitor-Service)
+
|ongoing (latest release 2.1, https://github.com/MfN-Berlin/BioCASe-Monitor-Service)
−
|ongoing (latest release version 3.6.3 is http://ww2.biocase.org/svn/bps2/branches/stable )
+
|ongoing (latest release version 3.8 is http://ww2.biocase.org/svn/bps2/branches/stable)
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
 
|ongoing (latest stable release see under [http://diversityworkbench.net/Portal/Software DWB Software])
−
|ongoing (latest stable release Version 4.1.1, http://cybertaxonomy.eu/download/taxeditor/stable/ released 07.12.2016)
+
|ongoing (latest stable release Version 5.24.0, http://cybertaxonomy.eu/download/taxeditor/stable/ released 02.06.2021)
−
| Vers. 2.5.1 (released January, 2016)  
+
| Vers. 3.4
−
| Vers. 3.3
+
| Productive System; development and project itself finished in 11/2014; ongoing maintainance at BGBM  
−
| Productive System; development and project itself finished in 11/2014;ongoing maintainance at BGBM  
+
 
|-
 
|-
 
|'''Code language, developer platform'''
 
|'''Code language, developer platform'''
Line 267: Line 248:
 
|C#, .Net Framework, since 2005
 
|C#, .Net Framework, since 2005
 
|Java, Eclipse
 
|Java, Eclipse
−
| Java, JSP
+
|Python
−
| Python
+
|Java, XQuery
−
| Java, XQuery
+
 
|-
 
|-
 
|'''User manual'''
 
|'''User manual'''
 
| -
 
| -
 
|[http://bacdive.dsmz.de/help/ Online manual]
 
|[http://bacdive.dsmz.de/help/ Online manual]
−
|Online API Doc available on Github [https://github.com/MfN-Berlin/BioCASe-Monitor-Service]
+
|Online API Doc available on [https://github.com/MfN-Berlin/BioCASe-Monitor-Service MfN GitHub]
 
|[http://wiki.bgbm.org/bps BPS Wiki]; [http://wiki.bgbm.org/bps/index.php/BeginnersGuide Beginners Guide]  
 
|[http://wiki.bgbm.org/bps BPS Wiki]; [http://wiki.bgbm.org/bps/index.php/BeginnersGuide Beginners Guide]  
 
|DD Manual as pdf file under [http://diversityworkbench.net/Portal/DWB_user_manuals DWB User Manuals]
 
|DD Manual as pdf file under [http://diversityworkbench.net/Portal/DWB_user_manuals DWB User Manuals]
Line 280: Line 260:
 
|DTN Manual as pdf file under [http://diversityworkbench.net/Portal/DWB_user_manuals DWB User Manuals]
 
|DTN Manual as pdf file under [http://diversityworkbench.net/Portal/DWB_user_manuals DWB User Manuals]
 
|Integrated into the help system of the software. Online available only older versions. Update is planned.  
 
|Integrated into the help system of the software. Online available only older versions. Update is planned.  
−
| Online manual ([https://knb.ecoinformatics.org/knb/docs/ Metacat Administration Guide])
 
 
| --
 
| --
 
| [http://wiki.bgbm.org/rebind_documentation/index.php/Best_Practice_Handbook Best Practice Handbook]
 
| [http://wiki.bgbm.org/rebind_documentation/index.php/Best_Practice_Handbook Best Practice Handbook]
Line 287: Line 266:
 
|on demand
 
|on demand
 
|on demand
 
|on demand
−
| self-explanatory, on demand if necessary; administration Manual on GitHub [https://github.com/MfN-Berlin/BioCASe-Monitor-Service]
+
|self-explanatory, on demand if necessary; administration manual on [https://github.com/MfN-Berlin/BioCASe-Monitor-Service MfN GitHub]
 
|on demand; [http://www.biocase.org/help_desk User Helpdesk]
 
|on demand; [http://www.biocase.org/help_desk User Helpdesk]
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; DD included since 2013; User Help Desk
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; DD included since 2013; User Help Desk
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; DD included since 2013; User Help Desk
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; DD included since 2013; User Help Desk
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; User Help Desk
 
|[http://www.snsb.info/Workshops.html DWB workshops] for users and database administrators; User Help Desk
−
|on demand; Testversion of [http://ws1.cybertaxonomy.org/ EDIT Demo-DB] with nightly reset, usable with current EDIT version; [http://cybertaxonomy.eu/download/workshop/mexico2016/EDIT-Workshop/Handout-EDIT-Platform-Workshop-v.1.01.docx Handout-EDIT-Platform-Workshop]
+
|on demand; testversion of [http://ws1.cybertaxonomy.org/ EDIT Demo-DB] with nightly reset, usable with current EDIT version; [http://cybertaxonomy.eu/download/workshop/mexico2016/EDIT-Workshop/Handout-EDIT-Platform-Workshop-v.1.01.docx Handout-EDIT-Platform-Workshop]
−
| -
+
 
| Workshops, individuals on demand
 
| Workshops, individuals on demand
 
|on demand
 
|on demand
Line 304: Line 282:
 
| management of resources (multimedia objects) related to items, descriptors and descriptor states
 
| management of resources (multimedia objects) related to items, descriptors and descriptor states
 
| interface for ingest of GFBio submission information packages (SIPs) (in planning stage)
 
| interface for ingest of GFBio submission information packages (SIPs) (in planning stage)
−
| -
 
 
| -
 
| -
 
| -
 
| -
Line 320: Line 297:
 
| -
 
| -
 
| +
 
| +
−
| -
 
 
| -
 
| -
 
| +
 
| +
Line 327: Line 303:
 
|BioCASe Wrapper V. 3.6.1, underlying db: PostgreSQL view
 
|BioCASe Wrapper V. 3.6.1, underlying db: PostgreSQL view
 
|BioCASe Wrapper V. 3.5, BacDive export under construction
 
|BioCASe Wrapper V. 3.5, BacDive export under construction
−
|not applicable as the BioCASe Monitor Service connects to serveral, independent BPS versions
+
|not applicable as the BioCASe Monitor Service connects to several, independent BPS versions
 
|
 
|
 
| -
 
| -
Line 333: Line 309:
 
| -
 
| -
 
|possible on demand
 
|possible on demand
−
| planned
 
 
| coming soon
 
| coming soon
 
| custom BioCASe Protocol Endpoint (indepentent of BioCASe Wrapper, full support of the BioCASe Protocol)
 
| custom BioCASe Protocol Endpoint (indepentent of BioCASe Wrapper, full support of the BioCASe Protocol)
Line 346: Line 321:
 
| +
 
| +
 
|in preparation (currently no priority)
 
|in preparation (currently no priority)
−
| -
 
 
| not planned
 
| not planned
 
| not planned
 
| not planned
Line 355: Line 329:
 
| -
 
| -
 
| -  
 
| -  
 +
| +
 
| -
 
| -
−
| coming soon
 
 
| -
 
| -
 
|in preparation (currently no priority)
 
|in preparation (currently no priority)
−
| yes
 
 
| not planned
 
| not planned
 
| Yes partly, see [http://wiki.bgbm.org/rebind_documentation/index.php/Ecologial_Metadata_Language EML for additional Metadata]
 
| Yes partly, see [http://wiki.bgbm.org/rebind_documentation/index.php/Ecologial_Metadata_Language EML for additional Metadata]
Line 372: Line 345:
 
| +
 
| +
 
|CDM-XML. Most data are available through WebServices, most of them as XML or JSON. Documentation available at http://cybertaxonomy.eu/cdmlib/rest-api.html
 
|CDM-XML. Most data are available through WebServices, most of them as XML or JSON. Documentation available at http://cybertaxonomy.eu/cdmlib/rest-api.html
−
| Transformations from EML to Dublin Core performed by Metacat OAI-PMH produce simple Dublin Core (DC).
 
 
| DC; RDF in development
 
| DC; RDF in development
 
| theoretically possible with minor additional adjustment/configuration
 
| theoretically possible with minor additional adjustment/configuration
Line 385: Line 357:
 
| + (export wizard)
 
| + (export wizard)
 
|DarwinCore-Archive (csv incl. metadata), Excel (for taxonomic core data)
 
|DarwinCore-Archive (csv incl. metadata), Excel (for taxonomic core data)
−
| -
 
 
| csv
 
| csv
 
| -
 
| -
Line 393: Line 364:
 
| -
 
| -
 
| -
 
| -
−
| -
+
| Can be run inside a Docker container.
−
|several document generators (html, MediaWiki format) and pipelines for dynamic web publication; local import and export of molecular sequence data in FASTA and FASTQ format, DELTA export for use by [http://www.navikey.net NaviKey] installations; see also [http://diversityworkbench.net/Portal/DWB_network_and_installations:_Real_life_examples DWB network and installations at SNSB]; [http://diversityworkbench.net/Portal/DWB_network DWB network solutions]; see also [http://diversityworkbench.net/Portal/DiversityDescriptions_Implementations DiversityDescriptions Quiz Version]
+
|several document generators (html, MediaWiki format) and pipelines for dynamic web publication; local import and export of molecular sequence data in FASTA and FASTQ format, DELTA export for use by [http://www.navikey.net NaviKey] installations; see also [http://diversityworkbench.net/Portal/DWB_network_and_installations:_Real_life_examples DWB network and installations at SNSB]; [http://diversityworkbench.net/Portal/DWB_network DWB network solutions]; DD is a suitable tool for conceptual schema development as done with the [http://www.mod-co.net/wiki/Schema_Representations MOD-CO schema]; see also [http://diversityworkbench.net/Portal/DiversityDescriptions_Implementations DiversityDescriptions Quiz Version]
 
| -
 
| -
 
|DTN managed open data (taxon reference lists and checklists) are in the DWB cloud at SNSB IT Center and available through DTN REST Web Service; documentation under http://services.snsb.info/DTNtaxonlists/rest/v0.1/static/api-doc.html  
 
|DTN managed open data (taxon reference lists and checklists) are in the DWB cloud at SNSB IT Center and available through DTN REST Web Service; documentation under http://services.snsb.info/DTNtaxonlists/rest/v0.1/static/api-doc.html  
 
|Remote editor included since Version 3.11; browser based editor (to improve usability) currently available for occurance data; development ongoing and close related to the overall development of the TaxonomicEditor.
 
|Remote editor included since Version 3.11; browser based editor (to improve usability) currently available for occurance data; development ongoing and close related to the overall development of the TaxonomicEditor.
−
| The establishment of a Metacat repository does not mean that the database system automatically has a connection to DataOne. SGN will manage and archive the GFBio compliant  EML-structured xml-files completely independently (though membership to DataOne might be considered.)
 
 
| -
 
| -
 
| -
 
| -
 
|}
 
|}
  
 +
Status: June 2021
 +
 +
----
  
 +
Back to [[Technical Documentations]]
  
−
Status: January 2017
+
[[Category:Technical Documentation of software used by GFBio Partners]]

Latest revision as of 17:20, 29 June 2021

Technical documentation of management systems, data processing and publication tools not specialised on collection data at the GFBio Collection Data Centers

One of the goals of GFBio is to strengthen the data centers at the Natural History Collections and Culture Collections in Germany and improve their infrastructure to manage, archive and publish biodiversity research data on the long run. As a consequence – latest after the realisation of the GFBio platform and federal infrastructure – they should be able (a) to mobilise their own data resources for research purposes and (b) to provide their technical infrastructure for managing, archiving and publishing biodiversity research data including multimedia data.

The tools and systems below are set up for management, processing and publishing of data from more than one data domain: collection and observation data, trait data, taxonomic, taxon reference and checklist data, sequence data, sampling data, survey data as well as scientific data packages as a whole. Some are browser-based applications with own online portal, others are usable as desk top stand-alone applications or are part of a client-server network. All are involved in the GFBio dataflow and part of the portfolios of GFBio data centers.

In parallel, the GFBio collection data centers documented their installations of collection management systems, multimedia data management systems, long-term archiving solutions and of GFBio related IT services, tools and databases at the data centers.

Database system, Homepage AQUiLA BacDive BioCASe Monitor Service (GFBio version) BioCASe Provider Software DiversityDescriptions (DD) DiversityProjects (DP) DiversityTaxonNames (DTN) EDIT Taxonomic Editor Morph-D-Base reBiND
General Information Installations at GFBio collection data centers/ archives SGN DSMZ MfN BGBM, DSMZ, MfN, SGN, SMNS, SNSB, ZFMK SMNS, SNSB, ZFMK SMNS, SNSB, ZFMK SMNS, SNSB, ZFMK BGBM ZFMK BGBM
Contact persons T. Winter, A. Penzlin and A. Allspach (SGN) L.Reimer (Mailcontact of BacDive Team: mailto:contact@bacdive.de) F. Glöckler (MfN) J. Holetschek (BGBM) A. Link (SNSB) J.C. Monje (SMNS), T. Weibulat and D. Triebel (SNSB) J.C. Monje (SMNS), D. Triebel (SNSB) and P. Grobe (ZFMK) A. Müller and K. Luther (BGBM) L. Vogt (Uni Bonn) and P. Grobe (ZFMK) D. Fichtmüller (BGBM)
Developer group, country Biodiversity Informatics SGN DSMZ IT Services & Bioinformatics Group Department of Science Data Management at the MfN Berlin, Germany Biodiversity Informatics BGBM, Germany SNSB IT Center Germany (together with MfN and UBT as far as the whole DWB platform is concerned) SNSB IT Center Germany (together with MfN and UBT as far as the whole DWB platform is concerned) SNSB IT Center Germany (together with MfN and UBT as far as the whole DWB platform is concerned) BGBM (and Naturalis, NL implementing the Validation Framework) Biodiversity Informatics ZFMK Biodiversity Informatics BGBM, Germany
Data domains collection data, collection management data, observation data scientific collection data, trait data, physiological, morphological and environmental descriptions data mapped in BioCASe Provider Software collection data, collection management data, observation data triple-structured scientific data, e. g. any kind of descriptive or trait data, attribute data, molecular sequence data, sampling data; matrix data import; interactive key functions scientific project-specific metadata and data package-specific metadata, project settings and identifiers taxonomic data, taxon reference and checklist data collection data, taxonomic data (core functionality), molecular data, descriptive data, identification keys morphological descriptions, taxonomic data, character matrices; current project: ontology based descriptions collection data, collection management data, observation data
Organismic data all recent and fossil organisms groups; geological data Bacteria and Archaea (other groups of microorganisms may be in future releases) not applicable all recent and fossil organism groups all all all recent and fossil organism groups all recent and fossil organism groups (potentially) all all recent and fossil organism groups
Software and Database System Server Operating System Ubuntu, Apache Tomcat Ubuntu 12.04 (Apache) currently on Linux, generally supported by any OS running Apache, SQLite and PHP MS Windows, Linux, Unix, Mac (any OS that runs Python 3); Also available as Docker image. MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10) MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10) MS Windows Server 2012 R2, 2016; (clients from MS Windows 7 to MS Windows 10) Linux, Mac OS, MS Windows Linux version 2.6. currently on Linux, generally supported by Mac OS, MS Windows as well
Database system PostgreSQL MySQL 5.1 SQLite MS SQL Server, MySQL, Access, Sqlite, Oracle, Postgres, Firebird, Sybase; Excel, CSV MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express MS SQL-Server 2012 to MS SQL-Server 2016 or MS SQL-Server Express MySQL, PostgreSQL, H2, SQL Server MySQL, ZOPE ODB eXist-db
User rights management Granular user rights assignment Read rights global, write individual Granular user rights assignment On DB level MS SQL-Server specific; few DWB specific features and roles added MS SQL-Server specific; few DWB specific features and roles added MS SQL-Server specific; few DWB specific features and roles added Roles can be assigned to users according to taxonomic groups and data types (e.g. distribution data only). Project admins have rights to edit all types of data. The underlying software library allows very granular user rights assignment (not yet implemented in the user interface) Group based user rights. Entries can be shared read-only or writable within groups. Released entries readable by the public Group based user rights managment through the eXist-db
Client Webapplication, Frontend every common webbrowser Webapplication, Frontend every common webbrowser Webapplication, Frontend every common webbrowser Webbrowser C# desktop application (rich client) (and Web API for the LIASlight project under construction) C# desktop application (rich client) C# desktop application (rich client) desktop application (rich-client), browser based client (for less complex operations, in development) Browser based client any modern browser
GIS functionalities PostGIS extension, Mapserver (WMS/WFS) - not applicable not applicable - - - Visualization of distribution and point maps None None
GUIs for data import generic import tool - web forms for metadata entry, metadata retrieval from BioCASe Provider Software - Import wizard (DELTA, SDD, CSV), matrix wizard, local import (FASTA, FASTQ) generic import tool; CSV generic import tool, import wizard; CSV Currently very simple (but more complex UIs planned) direct into the database, bulk upload of images possible Admin interface of eXist-db allows for GUI based upload of files and directory managment
GUIs for data export/ reports EXCEL, CSV CSV,PDF, via webservice: XML, JSON GUI for API usage (JSON) ABCD and DwC archive export Export (DELTA, SDD, EML, CSV, FASTA, FASTQ) various export tools, export wizard (coming soon) various export tools (e. g. for hierarchy, for accepted names etc.), export wizard; CSV Currently very simple (but more complex UIs planned) CSV, RDF (in development) None
GUI language German, English English English English default: english (multilingual through translation tables) default: english (multilingual through translation tables) default: english (multilingual through translation tables) English and German (Easy to extend for other languages) English English
Open access - - Yes not applicable Yes, DD software download Yes, DP software download Yes, DTN software download Yes Yes Yes
Open source - - Yes, MfN GitHub Yes, BPS SVN code repository Yes, DWB SVN code repository Yes, DWB SVN code repository Yes, DWB SVN code repository Yes, http://cybertaxonomy.eu/taxeditor/source-repository.html Available on request Yes, Details about reBiND Installation, Prerequisites and SVN code repository
Licenses - - GPL v.3 Mozilla Public License Version 1.1 GPL v.2 GPL v.2 GPL v.2 Mozilla Public License Version 1.2 (Project License) GPL Mozilla Public License (MPL) 2.0
Information model online BacDive model not applicable not applicable DD data model DP data model DTN data model Common Data Model (CDM) http://cybertaxonomy.eu/cdm/latest/ Coming soon No
State of development ongoing; beta version (search portal) online: https://search.senckenberg.de/aquila-public-search/search, https://www.senckenberg.de/en/science/research-infrastructure/collections/digitale-ressourcen/aquila/ ongoing ongoing (latest release 2.1, https://github.com/MfN-Berlin/BioCASe-Monitor-Service) ongoing (latest release version 3.8 is http://ww2.biocase.org/svn/bps2/branches/stable) ongoing (latest stable release see under DWB Software) ongoing (latest stable release see under DWB Software) ongoing (latest stable release see under DWB Software) ongoing (latest stable release Version 5.24.0, http://cybertaxonomy.eu/download/taxeditor/stable/ released 02.06.2021) Vers. 3.4 Productive System; development and project itself finished in 11/2014; ongoing maintainance at BGBM
Code language, developer platform Java, JavaServer Faces, PrimeFaces PHP, Python, JavaScript PHP, XSLT, Javascript Python, JavaScript C#, .Net Framework, since 2012 C#, .Net Framework, since 2013 C#, .Net Framework, since 2005 Java, Eclipse Python Java, XQuery
User manual - Online manual Online API Doc available on MfN GitHub BPS Wiki; Beginners Guide DD Manual as pdf file under DWB User Manuals DP Manual as pdf file under DWB User Manuals DTN Manual as pdf file under DWB User Manuals Integrated into the help system of the software. Online available only older versions. Update is planned. -- Best Practice Handbook
Training on demand on demand self-explanatory, on demand if necessary; administration manual on MfN GitHub on demand; User Helpdesk DWB workshops for users and database administrators; DD included since 2013; User Help Desk DWB workshops for users and database administrators; DD included since 2013; User Help Desk DWB workshops for users and database administrators; User Help Desk on demand; testversion of EDIT Demo-DB with nightly reset, usable with current EDIT version; Handout-EDIT-Platform-Workshop Workshops, individuals on demand on demand
Notes - - - - management of resources (multimedia objects) related to items, descriptors and descriptor states interface for ingest of GFBio submission information packages (SIPs) (in planning stage) - - - Several data quality modules available to check for common mistakes in ABCD files.
Interfaces to export data in various schemas and standards xml according ABCD schema - - + + - + - + - +
BioCASe Wrapper Version installed, description of dataflow BioCASe Wrapper V. 3.6.1, underlying db: PostgreSQL view BioCASe Wrapper V. 3.5, BacDive export under construction not applicable as the BioCASe Monitor Service connects to several, independent BPS versions - - - possible on demand coming soon custom BioCASe Protocol Endpoint (indepentent of BioCASe Wrapper, full support of the BioCASe Protocol)
xml according DarwinCore schema - under construction planned for a future version + - - + in preparation (currently no priority) not planned not planned
xml according EML schema - - - - + - - in preparation (currently no priority) not planned Yes partly, see EML for additional Metadata
xml according other schemas - + not yet, but in future version: any schema supported by the BPS LIDO; arbitrary xml schemas can be handled SDD-XML + + CDM-XML. Most data are available through WebServices, most of them as XML or JSON. Documentation available at http://cybertaxonomy.eu/cdmlib/rest-api.html DC; RDF in development theoretically possible with minor additional adjustment/configuration
txt, CSV export + + - + (DwC-Archive) + + + (export wizard) DarwinCore-Archive (csv incl. metadata), Excel (for taxonomic core data) csv -
Notes - - - Can be run inside a Docker container. several document generators (html, MediaWiki format) and pipelines for dynamic web publication; local import and export of molecular sequence data in FASTA and FASTQ format, DELTA export for use by NaviKey installations; see also DWB network and installations at SNSB; DWB network solutions; DD is a suitable tool for conceptual schema development as done with the MOD-CO schema; see also DiversityDescriptions Quiz Version - DTN managed open data (taxon reference lists and checklists) are in the DWB cloud at SNSB IT Center and available through DTN REST Web Service; documentation under http://services.snsb.info/DTNtaxonlists/rest/v0.1/static/api-doc.html Remote editor included since Version 3.11; browser based editor (to improve usability) currently available for occurance data; development ongoing and close related to the overall development of the TaxonomicEditor. - -

Status: June 2021

Back to Technical Documentations

Retrieved from "https://gfbio.biowikifarm.net/w//index.php?title=Technical_documentation_of_management_systems,_data_processing_and_publication_tools_not_specialised_on_collection_data&oldid=2673"