v1.3.0

Added

Features

• Data.dump_browse() and Data.show() now take an optional slice_axis
  keyword argument which controls which axis of a 3-D image array Data
  treats as the 'band' or 'channel' axis

Dataset Support

• MGS MOLA PEDR (a special case fixes non-standard STRUCTURE pointer names and
  'ROWS = "UNK"' from the labels)
• LRO LAMP EDR and most RDR table/image pointers
• additional IUE images
• Apollo Lunar Sample Photographs

Other

• Our JOSS paper was accepted for publication! 🎉 A pdf of our JOSS
  publication is included in the documentation. This is now our preferred
  citation. A CITATION.cff file is now included in the repo.

Changed

• DESCRIPTION pointers will now be deduplicated and print with indexes in the
  keys list (e.g. DESCRIPTION_0, DESCRIPTION_1, etc.). No warning will be triggered.
• Unit tests have been moved inside the pdr/ folder and will now be packaged
  and shipped with pdr
• Previously only pointers named exactly "^STRUCTURE" or "^PDS_OBJECT" skipped
  deduplication. Now all pointers with "STRUCTURE" in the name skip deduplication,
  and all pointers with "PDS_OBJECT" in the name get ignored.
• vendored pdr.vax module removed in favor of using SETI's rms-vax (successor
  package to that vendored module)
• pdr.pilutils renamed to pdr.pil_utils for consistency
• Data.get_scaled and functions that use it (like Data.dump_browse) now also
  mask nonfinite values (inf, -inf, and nan)
• pdr no longer greedily heads files when looking for PVL labels
• Added a check_special_block() call to Data.get_scaled

Fixed

• updated docs/LICENSE.md to match LICENSE.md
• pillow's DecompressionBombError now fully suppressed when reading desktop
  format images in primary mode

Removed

• pdr ignores all PDS3 pointers containing the string "PDS_OBJECT". Data
  previously returned a TBD message on attempts to access objects associated
  with these pointers. We have determined that these pointers were markers for
  internal software tools and do not refer to unique data objects.
• The text of our JOSS paper publication on pdr has been removed from
  joss_paper/paper.md. Please refer instead to the pdf under docs/pdr_joss_paper.pdf
• A warning is no longer raised for out-of-bounds values on converting to 8-bit
  for browsification

Full Changelog: v1.2.3...v1.3.0

v1.2.3

Changed

• Consolidated optional 'browsify' and 'desktop_image' dependencies into simply
  'pillow'.
• Any attempt to use functionality that requires an optional dependency now raises
  a ModuleNotFoundError naming the required module.
  • To support this, astropy objects are now always imported through
    loaders.astrowrap.
• Updates to documentation to include instructions for pip users, etc.

Fixed

• docs/index.md updated to reflect the same content as README.md

Removed

• Experimental save_sparklines() function in pdr.browsify, and along with
  it, the optional matplotlib dependency.
• Unnecessary full matplotlib dependency for Notebook use.

Full Changelog: v1.2.2...v1.2.3

v1.2.2

Added

Features

• Support for 'desktop' image formats in primary mode: JPEG, TIFF, GIF
  (still and animated), PNG, MPO/MPF, WebP (still and animated), JPEG2000/JP2,
  and BMP. Behavior is as follows:
  • pdr will attempt to parse all EXIF, TIFF tag, and MPO metadata, including
    embedded XML packets. Parsed metadata is available in Data.metadata and
    accessible with Data.metaget() and related methods.
  • Some basic image information is also available in Data.metadata (format,
    color mode, dimensions, MIME type, etc.)
  • Most files will have a single data object named 'IMAGE'.
  • Animated GIF and WebP images will have one data object per frame, named
    'FRAME_0', 'FRAME_1', etc.
  • MPO/MPF files (this includes many files with .jpg and .jpeg extensions)
    will have one data object per image. The MPO primary image is named 'IMAGE'.
    Subsequent images are assigned names based on their MPType and index within
    the file.
  • PDR automatically converts palette-mode images (like many GIFs) to 3D RGB
    arrays on load. In these cases, mappings between RGB values and original
    palette indices are available with data.metaget('palette').
  • GeoTiffs work like other TIFF files. pdr naively parses the GeoTiff
    metadata, but does not construct a CRS or otherwise derive projection
    information.
  • Note that some of PDR's convenience features like Data.show() do not
    work well with images that rely on transparency.

Dataset Support

• additional Cassini ISS calibration images
• MRO HiRISE RDRs and additional DTM images (were notionally supported, now have official support)

Other

• The text of our JOSS paper publication on pdr is now included under
  joss_paper/paper.md. It will be moved to docs/paper.md in next release.
• Clarifying comments were added to the supported_datasets documentation regarding
  spice files
• Community guidelines including a code of conduct and contribution guidelines
  have been added to the readme and docs folder

Changed

• Data.dump_browse() and .show() no longer apply default contrast enhancement
  if it would result in an image with only one value. Explicitly passing the
  image_clip argument will override this, if for some reason you want to.
• Levenshtein dependency is now optional. Users installing via pip will need to
  use the "fuzzy" option.
• tiff optional-dependency feature in has been changed to desktop_image for
  pip users to better fit with new features in this version

Fixed

• Data.dump_browse() now works correctly with FITS files opened in primary mode.

Removed

• CHANGELOG.md was removed in preference of solely docs/version_history.md
• Binder demo of example notebook due to issues with Binder servers.

Full Changelog: v1.2.1...v1.2.2

v1.2.1

Added

Features

• Full compatibility with numpy 2.0 for pds4 products
  (pds3 products already compatible in last release)

Changed

• pds4_tools is now vendored within pdr to apply changes for numpy 2.0
  compatibility and avoid hot patches

Removed

• pds4_tools dependency
• _patches module

Full Changelog: v1.2.0...v1.2.1

v1.2.0

Added

Features

• Compatibility with numpy 2.0 for pds3 products and fits pds4 products (non-fits PDS4 product numpy 2.0 compatibility awaits changes in pds4_tools dependency)
• Support for VAX_REAL data types in qubes
• pdr.Data.__contains__ method see issue #57
• Unit tests are now backed by pytest fixtures for improved flexibility and compatibility
• ROW_PREFIX_BYTES are now accounted for when calculating offset positions

Dataset Support

• Galileo NIMS qubes
• MSL
  • EMS EDRs and RDRs
  • additional navcam and hazcam RDRs
• Rosetta
  • CONSERT
  • ALICE supplementary tables
  • MIDAS RDRs
  • additional COSIMA tables
• LASP Mars datasets
• Venus Express (VEX)
• Phoenix
  • ASE EDRs, RDS
  • Telltale Experiment (TT) products
  • Atmospheric Opacity (AO) products
  • TEGA LED EDRs and SC RDRs
• MGN RSS solar wind experiment
• MGS RSS solar conjunction data
• MEX VMC EDRs and RDRs
• Mariner 10 MAG, PLS, and POS products
• WFF/ATM derived DEM images
• Misc additional products from:
  • Cassini RPWS, MIMI, and ISS
  • Galileo Probe, MAG, SSI
  • IHW
  • ground-based datasets at SBN and ATM
  • Stardust
  • LRO Mini-RF
  • NEAR NLR
  • ICE
  • Juno JADE
  • Pioneer Venus Orbiter OMAG
  • Vega
  • Voyager LECP
  • MEX PFS and MARSIS
  • MER navcam/hazcam/pancam RDRs

Other

• (GitHub-specific) CI for unit tests / coverage reports

Changed

• PDR now associates FITS HDUs with PDS data objects by matching byte ranges as described in PDS labels to HDU byte ranges as described in their FITS headers, rather than by performing fuzzy string match between HDU names and PDS object names
• PDS4 FITS files are now opened using astropy.io.fits on the backend instead of pds4_tools, mirroring the PDS3 FITS behavior
• Non-filled-out FITS cards are represented as None, rather than special astropy objects without stable byte representations
• Assorted backend refactoring, linting, and minor improvements
• Unit tests no longer unnecessarily write to the source tree
• All string objects loaded directly from products by Data ('DESCRIPTION', 'LABEL', etc.) canonicalize line endings to \n

Fixed

• More compatibility fixes for Python 3.9, mostly related to type annotations
• Improved reliability of associating compressed files with filenames that imply uncompressed files (and vice versa)
• Addressed some cases in which PDR wasn't accurately determining lengths for HEADER objects
• HISTOGRAM_IMAGE objects will now be read as images instead of attempting to be read as tables. (IMAGE_HISTOGRAMS will still be read as tables)
• setup.py also correctly pins numpy < 2.0.0
• The hotpatch for pds4-tools Python 3.12 incompatibility no longer has persistent side effects; for instance, conda will no longer complain about checksum mismatches.
• Bug reading PDS4-labeled FITS files

Removed

• pdr.Data.__iter__ method is being deprecated see issue #57
• pdr.Data.__getitem__ method is being deprecated see issue #57

Full Changelog: v1.1.2...v1.2.0

v1.1.2

[1.1.2] - 2024-06-18

Changed

• Pinned numpy version to less than 2.0 until compatibility issues with new
  release are resolved.

Full Changelog: v1.1.1...v1.1.2

v1.1.1

[1.1.1] - 2024-06-14

Fixed

• Bugfix for compatibility for Python 3.9 by adding two features of Python 3.10.
  • pdr.py: if inspect.get_annotations is not available, use a fallback
    implementation (incomplete, but works for the case we care about).
  • pdrtypes.py: if typing.TypeAlias is not available, define it by hand.

Full Changelog: v1.1.0...v1.1.1

v1.1.0

[1.1.0] - 2024-05-21

Added

Features

• Support for 4 byte VAX_REAL Tables
• Ability to modify Metadata objects and have changes propagate to loaded data.
  Suggested feature in issue #55

    Example Usage:
    Display an BSQ RGB image as a vertical image with 
    each channel split into a row
    
    data.metadata['IMAGE']['LINES'] = (
    data.metadata['IMAGE']['BANDS'] * data.metadata['IMAGE']['LINES']
    )
    data.metadata['IMAGE']['BANDS'] = 1
    data.load_metadata_changes()
    data.load('IMAGE', reload=True)
    data.show('IMAGE')

If the image was not loaded before the metadata change, the reload=True
argument is unnecessary.

Dataset Support

• Cassini UVIS EUV, FUV
• Voyager IRIS full-res spectral observations
• GRSFE AVIRIS and TIMS tables
• all GRSFE TABLE_HEADER and SPECTRUM_HEADER pointers
  • Affected datasets: AVIRIS and TIMS tables, PARABOLA, and wind experiment

Changed

• Updated several unit tests based on the Metadata changes

Fixed

• Bug which prevented reading of single band images with an interleave key if
  the interleave was BIL and the image had prefix/suffix. Resolves issue
  #55
• Bug in which some pds4 data objects were not being successfully cast from
  PDS4tools arrays to numpy arrays
• Bug in which padding bytes before top level containers in ASCII tables not
  described by a column in the label were not being respected. See below for
  description of this type of label writing practice.

  michael: yeah. the problem is that this isn't actually wrong, it's just depraved.
  it violates only the spirit of the law
  

v1.0.7

[1.0.7] - 2024-04-23

Added

Features

• pdr.fastread(), a convenience function for opening a product without
  checking for a detached label. This function only works if you point it at
  the file that contains the product's label (attached or detached). Improves
  speed on slow filesystems or for large numbers of products, particularly
  when you are only interested in their metadata or the data objects you want
  to load from them are small.

Dataset Support

• MEX SPICAM and MRS datasets
• Comet D/Shoemaker-Levy 9/Jupiter Impact Observing Campaign datasets
• see supported_datasets.md for details

Other

• Lots of docstrings, type hinting, and other in-code documentation
• Additional shared types to support static analysis and code readability
• pdr-tests-compatible annotations for special cases

Changed

• Assorted code refactoring, linting, and minor backend improvements
• Substantial performance increases for wide tables with many repeated items
• Modified some special cases for pandas 2.2.x compatibility

Fixed

• A bug in PDS4 label handling that sometimes dropped repeated child elements
• An accidentally-skipped out-of-date unit test for DSV tables
• Slow imports are again delayed until needed

Removed

• Unused numpy-based ASCII table parser

v1.0.6

Added

Features

• Data now directly affords a find_special_constants() method

Dataset Support

• most IHW datasets
• additional Voyager 1 and 2 datasets
• GRSFE and WFF/ATM datasets
• LOIRP dataset
• additional MSL, Apollo, and Mariner datasets
• additional Earth-based lunar data
• Viking Lander datasets
• Mars Odyssey DAN and SAM data
• MEX ASPERA, HRSC, SRC, and PFS data
• most Phoenix TEGA, ELEC, and WCL products
• see supported_datasets.md for details

Changed

• Assorted type-hinting and other in-code documentation improved.
• Behavior for character stripping in string columns of ASCII tables was
  inconsistent. To increase usability, string columns of ASCII tables now
  discard preceding and trailing spaces, commas, newlines, and double-quotes
  (regardless of strict label specification).
• Generally improved performance on table reading

Fixed

• Incorrect column bounds due to unusual prefix/suffix/infix specifications
  in some ASCII tables
• Incorrect column bounds due to container padding in some tables
• FITS HDUs with non-printable characters in header cards will now load,
  discarding any such cards from the header (unless the cards are required,
  e.g. NAXIS1). Resolves issue #52