Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.5 export harvest #3308

Merged
merged 226 commits into from
Aug 23, 2016
Merged
Changes from 1 commit
Commits
Show all changes
226 commits
Select commit Hold shift + click to select a range
98cad35
Merge branch '4.3' into 2579-DDI-export
sekmiller Jan 12, 2016
03c9fc8
Merge branch 'develop' into 2579-DDI-export
sekmiller May 9, 2016
0cf3ed0
Merge branch 'develop' into 2579-DDI-export
sekmiller May 12, 2016
b4def5a
built a local versio of xoai-serviceprovider, with a header parser bu…
landreev May 13, 2016
ee66db3
Merge branch 'develop' into 2579-DDI-export
sekmiller May 13, 2016
2c31855
Merge branch 'develop' into 2579-DDI-export
sekmiller May 16, 2016
0472396
Manage Harvesting Clients page expanded.
landreev May 17, 2016
0d8f741
Merge branch 'develop' into 2579-DDI-export
sekmiller May 18, 2016
b542811
Merge branch 'develop' into 2579-DDI-export
sekmiller May 18, 2016
165f5e1
More code for the harvesting client infrastructure.
landreev May 18, 2016
9c2636e
Fixed delete of a configured harvesting client on the manage clients …
landreev May 18, 2016
74200f3
populated the archive description of a newly created client with some…
landreev May 18, 2016
cff2942
Checking in the local copies of the patched XOAI service provider.
landreev May 18, 2016
7dce1e7
Fixed the default values for the weekly schedule (that was causing th…
landreev May 18, 2016
869300d
fixed week day order in the schedule.
landreev May 18, 2016
1e2d975
fixed validation for MetadataFormat (required);
landreev May 18, 2016
bd7a912
Fixed yet another bug in xoai-service provider.
landreev May 19, 2016
feab190
cleaned up and reorganized harvesterServiceBean - isolated the OAI-sp…
landreev May 19, 2016
5b1448a
added proper code for initializing an edit session for an existing cl…
landreev May 19, 2016
75a2b4b
View/edit existing harvesting clients.
landreev May 20, 2016
caabb5e
Initial clean up of the Manage Harvesting Clients pg. [ref #813]
mheppler May 26, 2016
e7357f2
Merge branch 'develop' into 2579-DDI-export
sekmiller May 27, 2016
e1ac44b
Merge branch 'develop' into 2579-DDI-export
sekmiller May 27, 2016
9b2edc7
A simple fix for the OaiClient constructor;
landreev Jun 7, 2016
63eac27
Added initial wireframe for the step-process workflow to Create Harve…
mheppler Jun 8, 2016
e560a34
Initial Check In - Support of Service Providers
sekmiller Jun 13, 2016
2c99fef
Cleaned up styling and layout of the Export Metadata button in the Me…
mheppler Jun 14, 2016
3553f48
Add more data elements
sekmiller Jun 15, 2016
7cade56
Fix Producers - remove dead code
sekmiller Jun 15, 2016
233da00
Add the rest of the DDI elements
sekmiller Jun 17, 2016
0ca567b
Refresh codebook.xsd
sekmiller Jun 17, 2016
386f9d7
Add Distributor and other DDI Elements
sekmiller Jun 21, 2016
964f314
Format Method Element
sekmiller Jun 21, 2016
ebf7eec
Add Elements to Test files
sekmiller Jun 21, 2016
070ed4e
Initial commit of the wireframe for the superuser dashboard. [ref #813]
mheppler Jun 21, 2016
3f6f482
More code, for server-side OAI-PMH functionality.
landreev Jun 21, 2016
531b59b
one more change:
landreev Jun 21, 2016
91882dc
Add Dublin Core Exporter Service
sekmiller Jun 22, 2016
dad2e86
Cleanup Export Service
sekmiller Jun 22, 2016
f8eef0f
Adding the parent XOAI pom file to the local_lib repository.
landreev Jun 22, 2016
12de6d5
Adding more Dublin Core Elements
sekmiller Jun 22, 2016
59850ce
Add Geo and Terms to Dublin Core
sekmiller Jun 22, 2016
cffb26b
Cleanup Dublin Core Export Util
sekmiller Jun 22, 2016
11f5f7d
Merge branch 'develop' into 813-harvesting
landreev Jun 22, 2016
e3b1d5d
these 2 files had merge conflicts as well; not sure why they didn't g…
landreev Jun 22, 2016
1cde62a
uh, gee.
landreev Jun 22, 2016
8a62bb5
another thing that didn't auto-merge properly...
landreev Jun 22, 2016
f5e809d
Wired up the breadcrumbs for the Manage Harvesting Clients pg to incl…
mheppler Jun 23, 2016
504dccd
More fixes to the harvesting code.
landreev Jun 28, 2016
88ed7be
Remove Debug code
sekmiller Jun 28, 2016
7e69715
Minor UI clean for page titles, help text, etc in the dashboard and h…
mheppler Jun 28, 2016
a70dbe6
Add Unit tests for Dub Core and DDI
sekmiller Jun 29, 2016
2395d71
Merge branch 'develop' into 2579-DDI-export
sekmiller Jun 29, 2016
f23bcdf
Format DDI Export
sekmiller Jun 30, 2016
10d3d89
Fix tag label
sekmiller Jul 5, 2016
ff1707d
Additional clean up of the UI for the harvestclients, harvestsets and…
mheppler Jul 5, 2016
96dcb14
Fix docDesc Block
sekmiller Jul 6, 2016
c1998a8
Update Test Files
sekmiller Jul 6, 2016
ef34efb
Merge branch 'develop' into 4.5-export-harvest
landreev Jul 6, 2016
7ec612c
Merge branch '2579-DDI-export' into 4.5-export-harvest
landreev Jul 6, 2016
5eaacb0
Add Logging to API remove from util
sekmiller Jul 7, 2016
d6fec65
Get export to work for migrated Datasets
sekmiller Jul 7, 2016
07a01dd
Add null check to Terms of Use items
sekmiller Jul 7, 2016
185059f
Assorted small changes/fixes for the harvesting;
landreev Jul 8, 2016
822e9e6
Cleaned up some minor layout issues, fixed some render bugs in the na…
mheppler Jul 8, 2016
10711b3
Fixed breadcrumbs bug on Manage Harvesting Clients and Manage Harvest…
mheppler Jul 8, 2016
e2139ac
split the "create new harvesting client" into 4 steps.
landreev Jul 8, 2016
fdea910
Couple of small fixes for the "edit existing client" mode.
landreev Jul 8, 2016
d4c4328
Add DDI export to Publish Dataset Command -
sekmiller Jul 8, 2016
3c1f752
Added empty help/info text with links to User Guide, fixed minor layo…
mheppler Jul 8, 2016
09f5ec0
Added the dataverse selection menu to the "add new client" wizard.
landreev Jul 8, 2016
870b2b2
One more bug fix in the validation code...
landreev Jul 8, 2016
58804ea
Fixed a bug in delete-a-set on the sets page.
landreev Jul 11, 2016
24c3b53
Fixed a bug with the "mother dataverse" for new harvesting client con…
landreev Jul 11, 2016
11df9be
Added an info message to show after a harvest gets started asynchrono…
landreev Jul 11, 2016
a861a3c
fixed the transaction attribute for the import-harvested-dataset method.
landreev Jul 12, 2016
8d3bb58
Rewrote the delete-harvested-dataset method using the Delete command.
landreev Jul 12, 2016
66e8563
Additional UI clean up for the harvesting workflow. [ref #813]
mheppler Jul 13, 2016
cc4c561
On Demand export open in new tab
sekmiller Jul 14, 2016
ced1ddd
Suppress On Demand Export for Draft
sekmiller Jul 14, 2016
6d0e089
Added unique placeholder text to the empty message on the Manage Harv…
mheppler Jul 14, 2016
71865f0
Made CreateDatasetCommand treat runtime exceptions in indexing as fatal;
landreev Jul 14, 2016
1d6e37b
Got rid of the static "403.xhtml" page redirects in the harvesting pa…
landreev Jul 14, 2016
454638b
harvesting info labels for the dashboard page.
landreev Jul 14, 2016
97e4512
Added code to cache the DDI export when a dataset is published.
landreev Jul 15, 2016
7ab2ecc
Additional minor UI clean up for Dashboard, Manage Clients, Manage Se…
mheppler Jul 15, 2016
6068576
Rewrote the export service, added caching of successful exports on th…
landreev Jul 18, 2016
13bb0e0
added a "get export as string" method to the export service.
landreev Jul 18, 2016
50c3899
a quick bug fix
landreev Jul 18, 2016
cf37301
Fixed success message for the sets page. Moved the texts for the succ…
landreev Jul 18, 2016
e782313
Added a proper "success" message, when enabling OAI service on the se…
landreev Jul 18, 2016
527149b
Harvesting: add "source" facet #3203
pdurbin Jul 18, 2016
f8241fc
Added methods to the dashboard page for individual elements of the ha…
landreev Jul 18, 2016
8b394d9
Rearranged the connection between Dataverse, (harvested) Dataset and …
landreev Jul 19, 2016
7240646
Fixed the spelling of the new "Harvested" solr field.
landreev Jul 19, 2016
468bc8c
minor fixes/cleanup of the harvester, import and harvestingclient ser…
landreev Jul 19, 2016
8c5c67a
added "deleted dataset count" to the harvest run statistics;
landreev Jul 20, 2016
43f023d
(added a comment to the exporter interface)
landreev Jul 20, 2016
2d681b4
Remove Export as Json Link from Metadata tab
sekmiller Jul 20, 2016
25eb1f0
Wired "name" and "affiliation" to the auto-complete Dataverse search …
landreev Jul 20, 2016
698a857
Cleaned up/rewrote the initial help tips for the "sets" page.
landreev Jul 20, 2016
c785d50
Added License to Json parser
sekmiller Jul 20, 2016
6c7d546
1 word fix for the harvest server help tip.
landreev Jul 20, 2016
4e677bf
Fixes for the sets page;
landreev Jul 21, 2016
8f882fb
The new exception for the OAI service...
landreev Jul 21, 2016
b731cb4
some fixes for the sets page.
landreev Jul 21, 2016
8cb9a7e
Fixes for the Sets datasets.
landreev Jul 21, 2016
92e89d0
Add Dataverse Theme to Json Parser/Printer
sekmiller Jul 21, 2016
efe1945
more cosmetic fixes for the sets page.
landreev Jul 21, 2016
ba007cf
Added datafile access API urls to the file sections in the DDI export...
landreev Jul 22, 2016
b7457b8
Undoing the uglier hacks I committed last night, in order to pass the
landreev Jul 22, 2016
9eb7049
Fixed broken tooltips on the Manage Clients and Manage Server pgs. Hi…
mheppler Jul 22, 2016
a72bb43
a quick fix for an error in oairecordservice introduced last night.
landreev Jul 22, 2016
2bc0bb9
Check for published version before export
sekmiller Jul 22, 2016
a4876e8
adding code for generating dataDscr sections. (not yet complete)
landreev Jul 23, 2016
ae64f51
Merge branch 'develop' into 4.5-export-harvest
landreev Jul 23, 2016
a1eb3f2
(compatibility change needed once merged with the current dev. branch…
landreev Jul 23, 2016
75f97f8
more conflicts after the dev branch merge.
landreev Jul 23, 2016
3289328
added export/harvest related database updates to the release update s…
landreev Jul 23, 2016
6e6b1ff
Added render logic and new labels to Dashboard pg. Added empty text m…
mheppler Jul 25, 2016
27a520a
Add notes for Dataset Export API call
sekmiller Jul 25, 2016
6979420
Add notes about Dataset Metadata Export Button
sekmiller Jul 25, 2016
ddd0274
Fixed the "add set" button when there are no sets configured;
landreev Jul 25, 2016
9d33c95
Changed the "agency" attribute of the <IDNo> ddi field to "handle" an…
landreev Jul 25, 2016
818a612
Fixed the "<IDNo>" in the ddi export (for reals this time!)
landreev Jul 26, 2016
043ac80
cosmetic fix to a delete warning in the bundle
landreev Jul 26, 2016
8d7ce31
Basic framework for importing files from harvested datasets (for clou…
landreev Jul 26, 2016
75dbca4
Fixed formatting of message text. [ref #813]
mheppler Jul 26, 2016
6fd32d3
Copy UNF when making new dataset version
sekmiller Jul 26, 2016
09db27f
Added code for importing and indexing harvested files.
landreev Jul 26, 2016
5ed2033
Cleaned up the layout of the Create/Edit Harvesting Set popup. [ref #…
mheppler Jul 27, 2016
cf8e581
silenced some super-noisy logging;
landreev Jul 27, 2016
d8084ba
Revised help text. [ref #813]
mheppler Jul 27, 2016
558d305
Fixed the problems with deleting harvesting clients with harvested fi…
landreev Jul 28, 2016
f5dec8d
Fixed the "add set" button; it shouldn't show until the service is en…
landreev Jul 28, 2016
0bac879
added:
landreev Aug 1, 2016
7082f91
checked in lots of .info logging messages for set updates (temporary);
landreev Aug 1, 2016
ecb92e6
Fixes for properly handling OAI deletes.
landreev Aug 1, 2016
199f253
Activate timers as soon as a scheduled harvest is created.
landreev Aug 1, 2016
c21f232
cleanup of a harvest-on-a-timer job.
landreev Aug 1, 2016
8d2c233
A fix for records in overlapping sets.
landreev Aug 2, 2016
266ba46
Code for removing harvested datafiles quickly and unceremoniously.
landreev Aug 2, 2016
d8f3960
A fix for the harvested dataset description in the search card.
landreev Aug 2, 2016
7db0518
Minor fixes:
landreev Aug 3, 2016
3a96e94
phantom change, to satisfy git stash pop.
landreev Aug 6, 2016
4e8c081
Adding missing code for harvesting of non-DDI formats;
landreev Aug 8, 2016
75b9e92
Remove required system setting of Exports on Demand
sekmiller Aug 8, 2016
f8d8508
Move Export Dataset instructions
sekmiller Aug 8, 2016
9888b44
Fixing the errors caused by the unintended merge with the swift branch!
landreev Aug 8, 2016
d9e53d9
Merge branch '4.5-export-harvest' of https://github.com/IQSS/datavers…
landreev Aug 8, 2016
bf23c25
Changed the flavor of the DC export to "DCTERMS", for the export frag…
landreev Aug 8, 2016
08692f2
added a comment on the differences between "dc" and "dcterms", and what
landreev Aug 8, 2016
bd1baec
An exporter for the DCTERMS (extended) version of DublinCore.
landreev Aug 8, 2016
a90cfd1
Fixed the global identifier issue in the DC harvest/import.
landreev Aug 8, 2016
dfcece1
Fixed and cleaned up DC export, import and harvest.
landreev Aug 9, 2016
f8aee6d
Fixed the "can't edit after a save" bug.
landreev Aug 9, 2016
96f3219
Changed 'Statistics' to 'Datasets' on the sets page.
landreev Aug 9, 2016
47597e5
Making sure there is a detailed explanation in the harvest log, if any
landreev Aug 9, 2016
c236855
fixed case in "and" and "or" in set definition queries.
landreev Aug 9, 2016
22a9d11
Added code for designating the timer server (in a multi-node cluster).
landreev Aug 9, 2016
04ab98f
Fixed varying heights of cards on dashboard. Moved Dashboard link int…
mheppler Aug 9, 2016
c8512e1
Added focus to first input fields of forms in Manage Harvesting Clien…
mheppler Aug 10, 2016
f389d65
Cleaned up the layout of the Manage Harvesting Clients/Server dataTab…
mheppler Aug 10, 2016
15ade7e
Fixed the JSON parser (it was throwing a null pointer on a file, if i…
landreev Aug 10, 2016
ff40b06
New "Mother Timer", that runs on the timer server and starts all othe…
landreev Aug 10, 2016
d959ce5
The harvest clients page no longer needs to create timers!
landreev Aug 10, 2016
9c18149
start an export timer automatically.
landreev Aug 10, 2016
afa6d27
Added new tooltips to Manage Harvesting Client/Server pgs. Moved some…
mheppler Aug 11, 2016
a1dab8a
Added a couple of extra query examples to the set definition help tex…
landreev Aug 11, 2016
0481692
Adding Dataset Language to Dublin Core Export #3271
sekmiller Aug 12, 2016
bc20300
Update Bundle.properties
djbrooke Aug 12, 2016
6d05c6a
minor changes to the bundle
landreev Aug 15, 2016
5268f0c
Adding the new harvesting/export guides that were missing!!
landreev Aug 15, 2016
848d3b3
Added the "30 characters" constraint to the set nickname validation o…
landreev Aug 15, 2016
b4c0988
Added alphabetical sort-by-name to the harvesting clients page, for c…
landreev Aug 15, 2016
9e73d86
a fix for disabling the oai service;
landreev Aug 15, 2016
ddf671c
Added code for deleting temp files left by the harvester.
landreev Aug 15, 2016
6cb164d
Removed the "filtering" step from populating the database with refere…
landreev Aug 15, 2016
170e084
Update harvestclients.rst
djbrooke Aug 15, 2016
9fa91fc
Update harvestclients.rst
djbrooke Aug 15, 2016
651e98c
Update harvestserver.rst
djbrooke Aug 15, 2016
c647f50
Update metadataexport.rst
djbrooke Aug 15, 2016
f87e9c6
#3281, #3276 - Add export log for export all errors
sekmiller Aug 15, 2016
92199f4
Changed the text on the button that adds a new set back to "Add Set".
landreev Aug 15, 2016
19d0723
removed html tags from a help message - they weren't getting rendered…
landreev Aug 15, 2016
361e858
Added explicit delete logic for the SOLR file documents belonging to …
landreev Aug 15, 2016
63ceb80
Extra rules for the DC impor (for the :publisher and :language fields)
landreev Aug 15, 2016
0d97c45
Added code to the "generic import service" to handle controlled vocab…
landreev Aug 16, 2016
28f4ee4
typo #813
pdurbin Aug 16, 2016
b4fd7ee
#3281 Add success message for each exported DS in batch job; and summ…
sekmiller Aug 16, 2016
a1df98c
Change Solr field from source to metadataSource, add isHarvested #3203
pdurbin Aug 16, 2016
fcf897f
Logic to disable the run, edit and delete buttons while a harvest is …
landreev Aug 16, 2016
64b8b51
Merge branch '4.5-export-harvest' of https://github.com/IQSS/datavers…
landreev Aug 16, 2016
8d93afb
added code to disable the 'delete' and other buttons while a delete i…
landreev Aug 16, 2016
36ad58f
Fix doubling up of Export Logging...
sekmiller Aug 16, 2016
0c3d6c6
Made the harvesting log messages only go to the dedicated log.
landreev Aug 16, 2016
153846f
Made the export log messages only go to the dedicated export log.
landreev Aug 16, 2016
8b4d29e
Code needed to force metadata reexport and OAI set updates when datas…
landreev Aug 17, 2016
a223191
A datacitation fix that makes it possible to re-index harvested datas…
landreev Aug 17, 2016
6726230
Update harvestserver.rst
djbrooke Aug 17, 2016
87011b2
Update harvestserver.rst
djbrooke Aug 17, 2016
29582c3
Fix for the "distributorName" in the citation for harvested datasets.
landreev Aug 17, 2016
7ba9194
A quick fix for DataCitation blowing up on ICPSR-registered DOIs
landreev Aug 17, 2016
5d4920d
A fixed for the cached dataverse name (I think)
landreev Aug 17, 2016
72c41af
Fixing my fix for the cached dataverse name from last night.
landreev Aug 18, 2016
d9da68d
Update export provider documentation
sekmiller Aug 18, 2016
25c8195
Changed the logging for exportAll, to log exceptions/failures.
landreev Aug 18, 2016
1a228f9
A nightly job that updates all the OAI sets will have its own log file.
landreev Aug 18, 2016
6fcb92d
Added a stern warning to the client delete confirmation message.
landreev Aug 19, 2016
e06738a
#3290 Remove [distributor] tag from citation for harvested datasets
sekmiller Aug 19, 2016
9315545
#3304 Add another fix for inconsistent data
sekmiller Aug 19, 2016
320afdc
Revised and styled confirmation msg in the Delete Harvesting Client p…
mheppler Aug 19, 2016
768b69b
fix for part 1. of #3296
landreev Aug 19, 2016
1d10570
Fixed CSS rule that was incorrectly applying min-width to content con…
mheppler Aug 19, 2016
59f20ae
more code for #3296
landreev Aug 19, 2016
b072c97
code for the second part of #3296
landreev Aug 19, 2016
63ff929
A quick fix for #3305; export and import for file (aka mime) type.
landreev Aug 22, 2016
300817f
Changes to the set and client delete messages, now that the deletes a…
landreev Aug 22, 2016
611aa0a
moved some code from the harvesting clients page and into the client …
landreev Aug 22, 2016
e1b6b2d
changed the version to 4.5
landreev Aug 22, 2016
b0b085b
Fixed a merge conflict syncing the export/harvest branch up to develop;
landreev Aug 22, 2016
2da2963
Toned down some verbose messages in harvest and index.
landreev Aug 22, 2016
74669d0
A couple of more unnecessary .info messages.
landreev Aug 22, 2016
a2899c8
Added a section on the application timers to the admin guide.
landreev Aug 23, 2016
fa46601
Added one more paragraph to the metadata export section of the guide, on
landreev Aug 23, 2016
722686a
Update timers.rst
kcondon Aug 23, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@
import edu.harvard.iq.dataverse.api.dto.FieldDTO;
import edu.harvard.iq.dataverse.api.dto.MetadataBlockDTO;
import edu.harvard.iq.dataverse.api.imports.ImportUtil.ImportType;
import static edu.harvard.iq.dataverse.export.ddi.DdiExportUtil.NOTE_TYPE_CONTENTTYPE;
import edu.harvard.iq.dataverse.util.StringUtil;
import java.io.File;
import java.io.FileInputStream;
@@ -1587,8 +1588,13 @@ private void processOtherMat(XMLStreamReader xmlr, DatasetDTO datasetDTO, Map fi
} else if ("id".equalsIgnoreCase(subjectType)) {
icpsrId = parseText(xmlr);
}
} else if (NOTE_TYPE_CONTENTTYPE.equalsIgnoreCase(noteType)) {
String contentType = parseText(xmlr);
if (!StringUtil.isEmpty(contentType)) {
dfDTO.setContentType(contentType);
}
}
}
}
} else if (event == XMLStreamConstants.END_ELEMENT) {// </codeBook>
if (xmlr.getLocalName().equals("otherMat")) {
// post process
@@ -1652,7 +1658,7 @@ else if (xmlr.getLocalName().equals("notes")) {
} else if ("id".equalsIgnoreCase(subjectType)) {
icpsrId = parseText(xmlr);
}
}
}
}
} else if (event == XMLStreamConstants.END_ELEMENT) {// </codeBook>
if (xmlr.getLocalName().equals("fileDscr")) {
@@ -1709,22 +1715,30 @@ private String processFileTxt(XMLStreamReader xmlr, FileMetadataDTO fmdDTO, Data
fmdDTO.setLabel( parseText(xmlr) );
/*sf.setFileType( FileUtil.determineFileType( fmdDTO.getLabel() ) );*/

} else if (xmlr.getLocalName().equals("fileType")) {
String contentType = parseText(xmlr);
if (!StringUtil.isEmpty(contentType)) {
dfDTO.setContentType(contentType);
}
} else if (xmlr.getLocalName().equals("fileCont")) {
fmdDTO.setDescription( parseText(xmlr) );
} else if (xmlr.getLocalName().equals("dimensns")) processDimensns(xmlr, dtDTO);
} else if (event == XMLStreamConstants.END_ELEMENT) {
if (xmlr.getLocalName().equals("fileTxt")) {
// Now is the good time to determine the type of this subsettable
// file (now that the "<dimensns>" section has been parsed, we
// should know whether it's a tab, or a fixed field:
String subsettableFileType = "application/octet-stream"; // better this than nothing!
if ( dtDTO.getRecordsPerCase() != null ) {
subsettableFileType="text/x-fixed-field";
} else {
subsettableFileType="text/tab-separated-values";
}
// If we still don't know the content type of this file
// (i.e., if there was no "<fileType>" tag explicitly specifying
// the type), we can try and make an educated guess. We already
// now that this is a subsettable file. And now that the
// "<dimensns>" section has been parsed, we can further
// decide if it's a tab, or a fixed field:
if (StringUtil.isEmpty(dfDTO.getContentType())) {
String subsettableFileType = "text/tab-separated-values";
if (dtDTO.getRecordsPerCase() != null) {
subsettableFileType = "text/x-fixed-field";
}
}
//EMK TODO: ask Gustavo & Leonid what should be used here instead of setFileType
// dfDTO.setFileType( subsettableFileType );
// dfDTO.setFileType( subsettableFileType );

return ddiFileId;
}
Original file line number Diff line number Diff line change
@@ -51,6 +51,9 @@
public class DdiExportUtil {

private static final Logger logger = Logger.getLogger(DdiExportUtil.class.getCanonicalName());

public static final String NOTE_TYPE_CONTENTTYPE = "DATAVERSE:CONTENTTYPE";
public static final String NOTE_SUBJECT_CONTENTTYPE = "Content/MIME Type";

public static String datasetDtoAsJson2ddi(String datasetDtoAsJson) {
logger.fine(JsonUtil.prettyPrint(datasetDtoAsJson));
@@ -1049,6 +1052,18 @@ private static void createOtherMats(XMLStreamWriter xmlw, List<FileDTO> fileDtos
xmlw.writeCharacters(fileDTo.getDataFile().getFilename());
xmlw.writeEndElement(); // labl
writeFileDescription(xmlw, fileDTo);
// there's no readily available field in the othermat section
// for the content type (aka mime type); so we'll store it in this
// specially formatted notes section:
String contentType = fileDTo.getDataFile().getContentType();
if (!StringUtilisEmpty(contentType)) {
xmlw.writeStartElement("notes");
writeAttribute(xmlw, "level", LEVEL_FILE);
writeAttribute(xmlw, "type", NOTE_TYPE_CONTENTTYPE);
writeAttribute(xmlw, "subject", NOTE_SUBJECT_CONTENTTYPE);
xmlw.writeCharacters(contentType);
xmlw.writeEndElement(); // notes
}
xmlw.writeEndElement(); // otherMat
}
}
@@ -1085,6 +1100,18 @@ private static void createOtherMatsFromFileMetadatas(XMLStreamWriter xmlw, List<
xmlw.writeCharacters(description);
xmlw.writeEndElement(); // txt
}
// there's no readily available field in the othermat section
// for the content type (aka mime type); so we'll store it in this
// specially formatted notes section:
String contentType = fileMetadata.getDataFile().getContentType();
if (!StringUtilisEmpty(contentType)) {
xmlw.writeStartElement("notes");
writeAttribute(xmlw, "level", LEVEL_FILE);
writeAttribute(xmlw, "type", NOTE_TYPE_CONTENTTYPE);
writeAttribute(xmlw, "subject", NOTE_SUBJECT_CONTENTTYPE);
xmlw.writeCharacters(contentType);
xmlw.writeEndElement(); // notes
}
xmlw.writeEndElement(); // otherMat
}
}