PyFRESCO Modules

RGBmap module for FRESCO package.

class pyfresco.RGBmap.RGBImageManipulator(img, img_sr, preset, ch1, ch2, ch3, wavelength)[source]

Bases: object

Class to generate the RGB map. With this class it is possible to manually control the contrast of each RGB channel and, after the RGB map is produced, it possible to save it both in .txt and .tiff and also to georeferentiate it.

Parameters:

  • img (python spectral.io.bsqfile.BsqFile object) – The spectral reflectance datacube.

  • img_sr (python spectral.io.bsqfile.BsqFile object) – The spectral parameter datacube.

  • preset (string) – The preset as given in Viviano-Beck et al., 2014 (https://doi.org/10.1002/2014JE004627). If None then no preset is selected.

  • ch1 (int) – If preset is None, this is the index of the spectral parameter used in the Red channel.

  • ch2 (int) – If preset is None, this is the index of the spectral parameter used in the Green channel.

  • ch3 (int) – If preset is None, this is the index of the spectral parameter used in the Blue channel.

  • wavelength (list or array) – The wavelengths detected by CRISM.

Labels()[source]
RGB_Viviano_Beck_2014(preset)[source]

This function uploads descriptions, spectral parameter names and combinations for RGB map as given in Viviano-Beck et al., 2014 (https://doi.org/10.1002/2014JE004627). Used in the function RGB_map_slider.

Parameters:

preset (string) – Pre-selected spectral parameters for RGB map generation as given in Viviano-Beck et al., 2014.

Returns:

  • names (list) – List of the spectral parameters corresponding to the RGB channels.

  • descriptions (string) – Description of the RGB map as given in Viviano-Beck et al., 2014.

  • indexes (list) – Indexes of the spectral parameters for the RGB map as they sorted in the spectral parameter datacube.

RGBmapmake(FALSE, bi, clip, cumhist, preset_true_colors, use_false_color, R_min_in=[0, 1], R_max_in=[0, 1], G_min_in=[0, 1], G_max_in=[0, 1], B_min_in=[0, 1], B_max_in=[0, 1], init_R=[0, 1], init_G=[0, 1], init_B=[0, 1], slider_step=0.005, slider_height=0.02, slider_width=0.25, slider_spacing=0.05)[source]

Function to perform the customization of the RGB map by moving apposite sliders to enhance constrast between different colors (i.e. spectral parameters). To finish the customization it is sufficent to close the plot window.

Parameters:

  • FALSE (3-dim array) – RGB image to be used as background.

  • bi (int) – Number of bins to divide the histograms into.

  • clip (bool) – If to clip the negative values or not.

  • cumhist (bool) – If to use cumulative histograms instead of frequency histograms.

  • preset_true_colors (string) – If use_false_color is False, here insert the preset name from Viviano-Beck et al., 2014 that wants to be used as background true color RGB image.

  • use_false_color (bool) – If to use a pre-computed RGB background map or to select another one from Viviano-Beck et al., 2014 as it is (without stretching).

  • R_min_in (list of two floats) – Minimum and maximum possible values for the minimum of the Red channel. Default is [0,1].

  • R_max_in (list of two floats) – Minimum and maximum possible values for the maximum of the Red channel. Default is [0,1].

  • G_min_in (list of two floats) – Minimum and maximum possible values for the minimum of the Green channel. Default is [0,1].

  • G_max_in (list of two floats) – Minimum and maximum possible values for the maximum of the Green channel. Default is [0,1].

  • B_min_in (list of two floats) – Minimum and maximum possible values for the minimum of the Blue channel. Default is [0,1].

  • B_max_in (list of two floats) – Minimum and maximum possible values for the maximum of the Blue channel. Default is [0,1].

  • init_R (list of two floats) – Initial values of the Red channel. Default is [0,1].

  • init_G (list of two floats) – Initial values of the Green channel. Default is [0,1].

  • init_B (list of two floats) – Initial values of the Blue channel. Default is [0,1].

  • slider_step (float) – Minimum step done by the slider while interacting with it. Default is 0.005.

  • slider_height (float) – Height at which the sliders are posed in the plot. Default is 0.02.

  • slider_width (float) – Width of the sliders. Default is 0.25.

  • slider_spacing (float) – Space between the sliders. Default is 0.05.

Returns:

  • RGB (3-dim array) – Final RGB map in the form of a numpy array.

  • stretches (list of float) – Final stretch values in the following order: final_min_R , final_min_G , final_min_B , final_max_R , final_max_G , final_max_B.

area(hist, bins, line1, line2)[source]

Function to calulate the percentile area of a histogram between two lines. Only used inside RGB_map_slider.

Parameters:

  • hist (array) – Histogram of the value of the RGB channel.

  • bins (int) – Number of bins of the histogram.

  • line1 (matplotlib.axes.Axes) – The line object of the left percentiles.

  • line2 (matplotlib.axes.Axes) – The line object of the right percentiles.

Returns:

Areas – List containing the area of the histogram on the left of the first line and on the right of the second line.

Return type:

list

f(RGB, min_R, min_G, min_B, max_R, max_G, max_B, clip=False)[source]

This function uploads the RGB map during the customization. This function is only used inside RGB_map_slider.

Parameters:

  • RGB (3-dim array) – The RGB image to be updated.

  • min_R (float) – Minimum value for the Red channel.

  • min_G (float) – Minimum value for the Green channel.

  • min_B (float) – Minimum value for the Blue channel.

  • max_R (float) – Maximum value for the Red channel.

  • max_G (float) – Maximum value for the Green channel.

  • max_B (float) – Maximum value for the Blue channel.

  • clip (bool) – If True it clips the negative values. Default if False.

Returns:

RGB_raw – Updated RGB map

Return type:

3-dim array

georeference(wkt, name, folder, min_lat, max_lat, westernmost_lon, easternmost_lon)[source]

Function to georeference a .tif image to a given reference frame.

Parameters:

  • wkt (string) –

    Reference system in which the image has to be georeferenced. Example for Mars –> wkt = “GEOGCRS[“GCS_Mars_2000”,DATUM[“D_Mars_2000”,

    ELLIPSOID[“Mars_2000_IAU_IAG”,3396190,169.894447223612, LENGTHUNIT[“metre”,1]]],PRIMEM[“Reference_Meridian”,0, ANGLEUNIT[“degree”,0.0174532925199433]],CS[ellipsoidal,2], AXIS[“geodetic latitude (Lat)”,north,ORDER[1], ANGLEUNIT[“degree”,0.0174532925199433]], AXIS[“geodetic longitude (Lon)”,east,ORDER[2], ANGLEUNIT[“degree”,0.0174532925199433]], USAGE[SCOPE[“unknown”],AREA[“World”], BBOX[-90,-180,90,180]],ID[“ESRI”,104905]]”

  • name (string) – Name of the .tif image you want to georeference.

  • fodler (string) – Path of the image you want to save.

  • min_lat (float) – Southernmost latitude of the image.

  • max_lat (float) – Northernmost latitude of the image.

  • westernmost_lon (float) – Westernmost longitude of the image.

  • Easternmost_lon (float) – Easternmost longitude of the image.

Return type:

None

savemap(name, folder=None, extension=None, show=False)[source]

Function to save the RGB map into 3 separated .txt files (one for each channel) and, if given, also in a specific image format.

Parameters:

  • name (string) – Name with which the RGB map is saved.

  • folder (string) – Path of the folder in which the RGB map is saved. If None it saves in home directory. Default is None.

  • extension (string, '.tif' or '.png' or '.jpg' or '.pdf') – Extension in which to save the RGB map as an image. If None it does not save it as an image. Default is None.

  • show (bool) – To show or not the resulting RGB map saved as an image. Deafult is None.

Return type:

None

pyfresco.RGBmap.open_raw(path_img_IF, path_hdr_IF, path_img_SR, path_hdr_SR)[source]

Function to open the spectral parameters and spectral reflectance datacubes.

Parameters:

  • path_img_IF (string) – Complete path of the reflectance datacube.

  • path_hdr_IF (string) – Complete path of the reflectance datacube header.

  • path_img_SR (string) – Complete path of the spectral parameters datacube.

  • path_hdr_SR (string) – Complete path of the spectral parameter datacube header.

Returns:

  • img (python spectral.io.bsqfile.BsqFile object) – Spectral reflectances datacube.

  • img_sr (python spectral.io.bsqfile.BsqFile object) – Spectral parameters datacube.

  • wavelength (list) – List of the CRISM observation wavelengths.

  • sr_names (list) – List of the CRISM spectral parameters.

SpectraExtract module

class pyfresco.SpectraExtract.SpectraExtract(img, Nbands, wavelength, MIN, MAX)[source]

Bases: object

Class to extract the spectra by selecting pixels or collections of pixels on a given RGB map.

Parameters:

  • img (python spectral.io.bsqfile.BsqFile object) – Spectral reflectances datacube.

  • Nbands (int) – Number of wavelengths used (basically, len(wavelength)).

  • wavelength (list or array) – CRISM observed wavelengths.

  • MIN (float) – Minimum wavelength value to be taken into consideration.

  • MAX (float) – Maximum wavelength value to be taken into consideration.

final_spectra(mean=True, size=[5, 15], c='blue')[source]

Function to compute and plot the final version of the spectra (mean +- standard deviation or median +- MAD).

Parameters:

  • mean (bool) – If True it computes and plots the mean and the standard deviation, if False it computes and plots the median and the MAD. Default is True.

  • size (list of two float) – Size of the plot.

  • c (string) – Color of the plot.

Returns:

  • m_spec (array) – Array of mean or median spectrum.

  • err_spec (array) – Array of the spectrum’ standard deviation or MAD.

find_nearest(array, value)[source]

Function used to find the index of element nearest to a given arbitrary value.

Parameters:

  • array (array) – Array in which to search

  • value (float) – Value to search the nearest element inside array.

Returns:

idx – Index of the nearest value.

Return type:

int

limits(other_w=None)[source]

Function to compute the limits of the x-axis and the corresponding indexes to then compute the y-axis limits of an array given the list/array of the x axis.

Parameters:

other_w (list, array or None) – If list or array, this will be taken as the wavelength array to use, if None then the CRISM observation wavelengths are used. Default is None.

Returns:

extremas – list of index correspondant do xmin , index correpsondant to xmax, xmin and xmax

Return type:

list of float

plot_spectra(N, ylim_min=0, ylim_max=0.5)[source]

Function to plot the spectra of the spectra arrays.

Parameters:

  • N (int) – Number of spectra

  • ylim_min (float) – Minimum y value from which to plot

  • ylim_max (float) – Maximum y value from which to plot

Return type:

None

point_spectra(N, save_pixel=False, folder=None, name='spectra_coordinates')[source]

Function to select pixels from which to extract the spectra by selecting points on the RGB image. The spectra are then extracted from the enclosed pixels from the spectral reflectances datacube.

Parameters:

  • N (int) – Number of points that wants to be taken.

  • save_pixel (bool) – If to save or not the enclosed pixels coordinates in a .txt file. Default is False.

  • folder (string) – Folder in which to save the pixels if save_pixels is True. If None it saves in the home directory. Must end with the /. Default is None.

  • name (string) – Name with which the pixels coorinates are saved. Default is ‘spectra_coordinates’.

Returns:

spectra – Spectra extracted from the spectral reflectance datacube from the pixels corresponding to the points drewn on the RGB map.

Return type:

2-dim array

polygon_spectra(c_line='white', c_marker='white', save_pixel=False, folder=None, name='spectra_coordinates')[source]

Function to select the area from which to extract the spectra by drawing a polygon on the RGB image. The spectra are then extracted from the enclosed pixels from the spectral reflectances datacube.

Parameters:

  • c_line (string) – Color of the polygon’s sides. Default is ‘white’.

  • c_marker (string) – Color of the polygon’s corners. Default is ‘white’.

  • save_pixel (bool) – If to save or not the enclosed pixels coordinates in a .txt file. Default is False.

  • folder (string) – Folder in which to save the pixels if save_pixels is True. If None it saves in the home directory. Must end with the /. Default is None.

  • name (string) – Name with which the pixels coorinates are saved. Default is ‘spectra_coordinates’.

Returns:

  • spectra (2-dim array) – Spectra extracted from the spectral reflectance datacube from the pixels enclosed in the polygon drewn on the RGB map.

  • L (int) – Amount of pixels selected.

  • mask (2-dim array) – Masked RGB array to enhance polygon position.

save_spectra(name, folder, method='polygon')[source]

Function to save the extracted spectra with either method and the cut wavelength range.

Parameters:

  • name (string) – Name of the file.

  • folder (string or None) – If None it will be saved into the home folder, if a folder path is given, the path must end with the /.

  • method (string) – It add a suffix standing for the used extraction method. Default is polygon.

Return type:

None

save_spectrum(name, folder, method='polygon', mean=True)[source]

Function to save the mean/median and std/MAD spectra and the cut wavelength range.

Parameters:

  • name (string) – name of the file.

  • folder (string or None) – If None it will be saved into the home folder, if a folder path is given, the path must end with the /.

  • method (string) – It add a suffix standing for the used extraction method. Default is polygon.

  • mean (Bool) – if True it will add a suffix with mean/std at the end of the filename, if False it will add median/mad instead.

Return type:

None

square_spectra(N, show=False, save_pixel=False, folder=None, name='spectra_coordinates')[source]

Function to select the area from which to extract the spectra by drawing a square on the RGB image. The spectra are then extracted from the enclosed pixels from the spectral reflectances datacube. The square is drewn by selecting the center of the square with the right mouse button and giving to it a side length. IMPORTANT: THE GIVEN SQUARE SIDE LENGTH MUST BE AN ODD NUMBER!

Parameters:

  • N (int) – Side length of the square (MUST BE AN ODD NUMBER!).

  • show (bool) – Whether or not to show the location of the selected square. Default is False.

  • save_pixel (bool) – If to save or not the enclosed pixels coordinates in a .txt file. Default is False.

  • folder (string) – Folder in which to save the pixels if save_pixels is True. If None it saves in the home directory. Must end with the /. Default is None.

  • name (string) – Name with which the pixels coorinates are saved. Default is ‘spectra_coordinates’.

Returns:

  • spectra (2-dim array) – Spectra extracted from the spectral reflectance datacube from the pixels corresponding to the points enclosed in the square on the RGB map.

  • x (list) – X coordinates of the pixels inside the square.

  • y (list) – Y coordinates of the pixels inside the square.

upload_map(name, folder=None)[source]

Function to upload a pre-made RGB map, saved using RGBImageManipulator.save_map(), that wants to be used to extract the spectra.

Parameters:

  • name (string) – Name of the RGB map that wants to be uploaded.

  • folder (string) – Path of the RGB map that want s to be uploaded. If None path is taken as home directory. Default is None.

Returns:

RGB – Uploaded RGB map.

Return type:

3-dim array

upload_spectrum(name, folder=None, mean=True)[source]

Function to upload a set of pre-extracted spectra, saved using SpectraExtarct.save_spectra().

Parameters:

  • name (string) – Name of the file that wants to be uploaded, without the format extension.

  • folder (string) – Path of the file that want s to be uploaded. If None path is taken as home directory. Default is None.

Returns:

  • spectra (2-dim array) – Uploaded pre-extracted spectra.

  • target_spectrum (1-dim array) – Mean/median of the pre-extracted spectra.

  • error_spectrum (1-dim array) – Standard deviation/median absolute deviation of the pre-extracted spectra.

SpectraNorm module

class pyfresco.SpectraNorm.SpectraNorm(RGB, Nbands, img, img_sr, wavelength, target, error_target, spectra, MIN, MAX)[source]

Bases: object

Class to Normalize the mean target spectrum.

Parameters:

  • RGB (3-dim array or None) – The RGB image to be used. Inside the class it also possible to upload the RGB map, so it is possible to set this parameter as None.

  • Nbands (int) – Number of wavelengths used (basically, len(wavelength)).

  • img (python spectral.io.bsqfile.BsqFile object) – Spectral reflectances datacube.

  • img_sr (python spectral.io.bsqfile.BsqFile object) – Spectral parameters datacube.

  • wavelength (list or array) – CRISM observed wavelengths.

  • target (array) – The mean target spectrum.

  • error_target (array) – The standard deviation of the target spectrum.

  • MIN (float) – Minimum wavelength value to be taken into consideration.

  • MAX (float) – Maximum wavelength value to be taken into consideration.

bootstrapnorm(convexhull, N=1000, interp='linear', lower=2.5, upper=97.5)[source]

Normalization of the target spectrum over the neutral. The error propagation is performed using a bootstrap algorithm.

Parameters:

  • convex_hull (bool) – If the normalization is done with the convex hull or not. Default is False.

  • N (int) – Number of bootstrap iteration. Deafult is 1000.

  • interp (string {‘linear’, ‘nearest’, ‘nearest-up’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’, or ‘next’}) – Is the interpolation of the convex hull using interp1d from scipy (https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html). The possible arguments signify: ‘zero’, ‘slinear’, ‘quadratic’ and ‘cubic’ refer to a spline interpolation of zeroth, first, second or third order; ‘previous’ and ‘next’ simply return the previous or next value of the point; ‘nearest-up’ and ‘nearest’ differ when interpolating half-integers (e.g. 0.5, 1.5) in that ‘nearest-up’ rounds up and ‘nearest’ rounds down. Default is ‘linear’.

  • lower (float) – Lower bound for the percentile calculation. Default is 2.5.

  • upper (float) – Upper bound for the percentile calculation. Default is 97.5

Returns:

  • norm (array) – Normalized spectrum.

  • error norm (array) – Propagated error of the normalized error. Calculated as mean of the upper and lower percentile bounds.

find_nearest(array, value)[source]

Function used to find the index of element nearest to a given arbitrary value.

Parameters:

  • array (array) – Array in which to search

  • value (float) – Value to search the nearest element inside array.

Returns:

idx – Index of the nearest value.

Return type:

int

limits(other_w=None)[source]

Function to compute the limits of the x-axis and the corresponding indexes to then compute the y-axis limits of an array given the list/array of the x axis.

Parameters:

other_w (list, array or None) – If list or array, this will be taken as the wavelength array to use, if None then the CRISM observation wavelengths are used. Default is None.

Returns:

extremas – list of index correspondant do xmin , index correpsondant to xmax, xmin and xmax

Return type:

list of float

mineral_mask(b, band_n, band_v)[source]

Function to obtain the neutral spectra from the mineral mask method used and defined by Horgan et al., 2020 (https://www.sciencedirect.com/science/article/pii/S0019103518306067).

It works by selecting a set of spectral parameters and selecting pixels for the neutral spectrum by imposing threshold values on the parameters themselves with the following rules:

\[\begin{split}\begin{aligned} &\text{If reflectance:} \quad \begin{cases} R... \geq \text{threshold} &\Rightarrow \text{Pixel in mineral mask} \\ R... \leq \text{threshold} &\Rightarrow \text{Pixel outside the mineral mask} \end{cases} \\ &\text{If band depth (BD):} \quad \begin{cases} BD... \leq \text{threshold} &\Rightarrow \text{Pixel in mineral mask} \\ BD... \geq \text{threshold} &\Rightarrow \text{Pixel outside the mineral mask} \end{cases} \end{aligned}\end{split}\]

Where R… are the pure reflectance parameters, and BD… symbolizes all other spectral parameter types.

Parameters:

  • b (int) – Number of bins to divide the histograms into.

  • band_n (list of strings) – Names of bands to use in the mineral mask. Accepted names are those given in Viviano-Beck et al., 2014.

  • band_v (list or array of float) – Threshold values for each band.

Returns:

  • neutral (2D array) – Array containing the neutral spectra.

  • med (1D array) – Median neutral spectrum.

  • mad (1D array) – MAD of the neutral spectrum.

moving_average(window_size, limiti=True)[source]

Function to perform a moving average smoothing on the normalized spectrum.

Parameters:

window_size (int) – Size of the step taken for the moving mean.

Returns:

result – Moving-mean smoothed normalized spectrum.

Return type:

1-dim array

neutral_all_map_multiple(RGBs_names, RGB_path, p, threshold, names)[source]

Function to obtain the neutral spectra with the so called ‘all map’ method consisting into creating an array of zeros and ones for each RGB image where the ones represents the points for that RGB with the three channel at 0 (+ a threshold), then this arrays are summed together and the neutral spectra is obtained by averaging all the spectra that corresponds to the pixels in which we have ones in all the maps (e.g if we have three maps, the final map will have only integer values of 0, 1, 2 or 3. We will take only the spectra corresponding to pixels of value 3 (or greater than a specified integer value) and we will do the average between them).

Parameters:

  • RGBs_names (list of strings) – The names of the RGB maps used.

  • RGB_path (string) – Path to the RGB maps used.

  • p (int) – Minimum number of maps or the number of image with zeros in common. IMPORTANT: CANNOT EXCEED THE NUMBER OF RGB MAPS GIVEN.

  • Threshold (float) – Threshold value to use if not so many zero pixels are avaliable in some maps.

  • names (list of strings) – Code names of the RGB maps for the plot of zero valued pixels per map.

Returns:

  • neutral (2-dim array) – Array containing the neutral spectra.

  • med (1-dim array) – Median neutral spectrum.

  • mad (1-dim array) – MAD of the neutral spectrum.

  • superimposed (2-dim array) – Array of the superimposed zero valued pixels per map. In this map 0 means no zeros in common on that pixel, 1 means that 1 map have a zero there, 2 means that two maps have a zero in common, etc.

  • zero_map (3-dim array) – Array of the zero pixels per map.

  • xx (list) – X coordinates of the pixels containing the neutral spectra.

  • yy (list) – Y coordinates of the pixels containing the neutral spectra.

neutral_all_map_single(threshold)[source]

Simpler version of SpectraNorm.neutral_all_map_multiple() that takes into account the zero-valued pixels of only one RGB map. In this case the used RGB map is the one given in the main or the one uploaded using SpectraNorm.upload_map().

Parameters:

Threshold (float) – Threshold value to use if not so many zero pixels are avaliable in some maps.

Returns:

  • neutral (2-dim array) – Array containing the neutral spectra.

  • med (1-dim array) – Median neutral spectrum.

  • mad (1-dim array) – MAD of the neutral spectrum.

  • x (list) – X coordinates of the pixels containing the neutral spectra.

  • y (list) – Y coordinates of the pixels containing the neutral spectra.

neutral_convex_hull(interp='linear')[source]

Function to select as neutral spectrum the convex hull (the calculation of is mutuated from scipy https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.ConvexHull.html).

Parameters:

interp (string {‘linear’, ‘nearest’, ‘nearest-up’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’, or ‘next’}) – Is the interpolation of the convex hull using interp1d from scipy (https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html). The possible arguments signify: ‘zero’, ‘slinear’, ‘quadratic’ and ‘cubic’ refer to a spline interpolation of zeroth, first, second or third order; ‘previous’ and ‘next’ simply return the previous or next value of the point; ‘nearest-up’ and ‘nearest’ differ when interpolating half-integers (e.g. 0.5, 1.5) in that ‘nearest-up’ rounds up and ‘nearest’ rounds down. Default is ‘linear’.

Returns:

  • neutral (1-dim array) – Points of the convex hull.

  • med (1-dim array) – Median of the convex hull, in this case is the same as neutral, but for completeness we keeped the same strucutre as the others methods.

  • mad (1-dim array) – MAD of the convex hull, since it only one “spectrum” it is an array filled with zeros.

neutral_polygon_spectra(c_line='white', c_marker='white', save_pixel=False, folder=None, name='neutral_spectra_coordinates')[source]

Function to select the area from which to extract the neutral spectra by drawing a polygon on the RGB image. The spectra are then extracted from the enclosed pixels from the spectral reflectances datacube. This function is essentially the same as SpectraExtract.polygon_spectra().

Parameters:

  • c_line (string) – Color of the polygon’s sides. Default is ‘white’.

  • c_marker (string) – Color of the polygon’s corners. Default is ‘white’.

  • save_pixel (bool) – If to save or not the enclosed pixels coordinates in a .txt file. Default is False.

  • fodler (string) – Folder in which to save the pixels if save_pixels is True. If None it saves in the home directory. Default is None.

  • name (string) – Name with which the pixels coorinates are saved. Default is ‘spectra_coordinates’.

Returns:

  • spectra (2-dim array) – Spectra extracted from the spectral reflectance datacube from the pixels enclosed in the polygon drewn on the RGB map.

  • med (1-dim array) – Median of the extracted spectra.

  • mad (1-dim array) – MAD of the extracted spectra.

  • L (int) – Amount of pixels selected.

  • mask (2-dim array) – Masked RGB array to enhance polygon position.

norm_spectra(convex_hull=False)[source]

Normalization of the target spectrum over the neutral. The error propagation formula is used for the resulting final error, and thus can lead to some places having complex error due to the presence of the covariance between the spectra. More advanced methods should be used to evaluate the error in those cases, but for most application it is sufficently good like this. Anyway, errors that ends up as complex are set to zero for simplicity.

Calling \(N\) the normalized spectrum, \(\sigma_N\) the resulting error of the normalized spectrum, \(A\) the target mean spectrum, \(B\) the median neutral spectrum, \(\sigma_A\) the standard deviation of the target, \(\sigma_B\) the MAD of the neutral, and \(C_{A,B}\) the covariance between the two spectra, the normalization is done in the following way:

\[\begin{split}\begin{aligned} N &= \frac{A}{B} \\ \sigma_{N} &= \left| \frac{A}{B} \right| \sqrt{ \left(\frac{\sigma_{A}}{B}\right)^{2} + \left(\frac{\sigma_{B}}{B}\right)^{2} - \frac{2C_{A,B}}{A \cdot B} } \end{aligned}\end{split}\]
Parameters:

convex_hull (bool) – If the normalization is done with the convex hull or not. Default is False.

Returns:

  • norm (array) – Normalized spectrum.

  • error norm (array) – Propagated error of the normalized error.

normplot(convex_hull=False)[source]

Function to plot the normalized spectrum.

Parameters:

None

Return type:

None

plot_together(convex_hull=False)[source]

Function to plot together the mean target spectrum +- its standard deviation and the median neutral spectrum +- its MAD.

Parameters:

convex_hull (bool) – If the convex hull neutral spectrum is used. This is done since the convex hull spectrum is drewn directly onto the x-cut spectrum and so does not need to be cut in the same range as the target. Default is False.

Return type:

None

save_spectrum(name, folder, method, normalized=True)[source]

Function to save the mean/median and std/MAD spectra and the cut wavelength range.

Parameters:

  • name (string) – Name of the file.

  • folder (string or None) – If None it will be saved into the home folder, if a folder path is given, the path must end with the /.

  • method (string) – It will add a suffix after the name on the base on the method with which the neutral spectra was computed. If method is not ply , sam , mam , min or cxh, this function will not work. ply stands for polygon, sam for single allmap, mam for mutiple allmap, min for mineral mask and csh for convex hull.

  • normalized (bool) – If True it will also save the normalize spectrum, if False not.

Return type:

None

savgol(window, order, limiti=True)[source]

Function to perform a Savitzky-Golay smoothing on the normalized spectrum as given in https://pubs.acs.org/doi/10.1021/ac60214a047. Mutuated from scipy.signal (https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.savgol_filter.html).

Parameters:

  • window (int) – Size of the convolution window.

  • order (int) – Order of the convolutional fit.

Returns:

result – Moving-mean smoothed normalized spectrum.

Return type:

1-dim array

upload_map(name, folder=None)[source]

Function to upload a pre-made RGB map, saved using RGBImageManipulator.save_map(), that wants to be used to extract the spectra.

Parameters:

  • name (string) – Name of the RGB map that wants to be uploaded.

  • folder (string) – Path of the RGB map that want s to be uploaded. If None path is taken as home directory. Default is None.

Returns:

RGB – Uploaded RGB map.

Return type:

3-dim array

upload_neutral(name, method, folder)[source]

Function to upload a pre-made median/mad spectrum.

Parameters:

  • name (string) – Name of the file.

  • method (string) – It will add a suffix after the name on the base on the method with which the neutral spectra was computed.

  • folder (string or None) – If None it will be saved into the home folder, if a folder path is given, the path must end with the /.

Returns:

  • med (1d-array) – The uploaded median spectrum.

  • mad (1d-array) – The MAD of the uplkoaded median spectrum.

  • w (1d-array) – The cut wavelength range.

upload_spectrum(name, folder=None, mean=True)[source]

Function to upload a set of pre-extracted spectra, saved using SpectraExtarct.save_spectra().

Parameters:

  • name (string) – Name of the file that wants to be uploaded, without the format extension.

  • folder (string) – Path of the file that want s to be uploaded. If None path is taken as home directory. Default is None.

Returns:

  • spectra (2-dim array) – Uploaded pre-extracted spectra.

  • target_spectrum (1-dim array) – Mean/median of the pre-extracted spectra.

  • error_spectrum (1-dim array) – Standard deviation/median absolute deviation of the pre-extracted spectra.

SpectraAnalysis module

class pyfresco.SpectraAnalysis.MaficAnalysis(spectrum, wavelength, MIN, MAX, fsize=15, pre_normed=True)[source]

Bases: object

Adpated from the paper from Horgan et al., 2014 ( https://doi.org/10.1016/j.icarus.2014.02.031 ). This class is taylored for the analysis of mafic materials and mixes.

Parameters:

  • data (numpy array) – Data (i.e. mean spectrum) to be analyzed.

  • wav (numpy array) – Wavelengths used as x-coordinate.

  • MIN (float) – Minimum wavelength value

  • MAX (float) – Maximum wavelength value

  • fs (float) – Font size of text of axis and legends in the plots.

band_parameters(windows_nm=75, resolution_nm=5, plot=False, tol=10, smoothed_after_removed=False)[source]

Core function of the class. Adapted from Horgan et al., 2014 (https://doi.org/10.1016/j.icarus.2014.02.031). The band parameters are the band minimum, band center, band depth, band area and band asymmetry. For more infor refer to the paper. The function can sometime not catch up and returns some error. Try with other parameters configuration in case. The issue refers to the way band centers are computed: due to the interpolation, sometimes that interpolation does not have a minium in the give region around the band minimum.

Parameters:

  • windows_nm (float) – Nanometric window around the band minimum where to search the band center.

  • resolution_nm (float) – Nanometric resolution of the 4-th order polynomial approximation used for the band center determination.

  • plot (bool) – Whether to plot the result. Default is False.

  • tol (float) – Minimum band wavelength span in nanometers. Bands with less wavelength span will not be analyzed.

Returns:

parameters – List of the parameters for each band.

Return type:

list

bands_removal(windows, mins, maxs, plot=False)[source]

Bands removal technique involving a moving average over a given subset(s) of the spectrum.

Parameters:

  • windows (list) – List of the smoothing window to apply to the subset(s).

  • mins (list) – List of the minimum values at which the smoothing process needs to be applied.

  • maxs (list) – List of the maximum values at which the smoothing process needs to be applied.

  • plot (bool) – Whether to plot the result. Default is False.

Returns:

spectrum – Cleaned spectrum. IMPORTANT, THIS OVERWRITES THE INPUT DATA.

Return type:

array

continuum_removal(interptype='linear', plot=False, smooth=False)[source]

Applies continuum removal using convex hull or flat model for out-of-range spectra. The hull will be constructed from the beginning to the maximum point of the spectrum. After that the hull will be a flat line.

Parameters:

  • interptype (string) – Interpolation type for continuum modeling

  • plot (bool) – Whether to plot the result. Default is False.

  • smooth (bool) – Whether the input is the smoothed spectrum or not. Default is False.

Returns:

removed_data – Continuum-removed reflectance.

Return type:

array

find_nearest(array, value)[source]
moving_average(window_size, plot=False, overwrite=False, removed=True)[source]

Function to perform a moving average smoothing on the normalized spectrum.

Parameters:

  • window_size (int) – Size of the step taken for the moving mean.

  • plot (bool) – Whether to plot the result. Default is False.

  • overwrite (bool) – Whether or not to overwrite the previsou self.data. Default is False.

  • removed (bool) – Whether or not the data underwent continuum removal process or not. Default is True.

Returns:

result – Moving-mean smoothed normalized spectrum.

Return type:

array

plot()[source]
class pyfresco.SpectraAnalysis.SpectraAnalysis(final, final_error, m_spec, err_spec, n_spectra, n_err, wavelength, MIN, MAX, folder='pyfresco/data')[source]

Bases: object

Class to perform the required spectral analyses.

Parameters:

  • final (1-dim array) – Normalized spectrum as given by SpectraNorm.norm_spectra().

  • m_spec (1-dim array) – Mean target spectrum.

  • err_spec (1-dim array) – Standard deviation of the target spectrum.

  • n_spectra (1-dim array) – Median neutral spectrum.

  • n_err (1-dim array) – MAD of the neutral spectrum.

  • wavelength (list or array) – CRISM observation wavelengths.

  • MIN (float) – Minimum wavelength value to be taken into consideration.

  • MAX (float) – Maximum wavelength value to be taken into consideration.

  • folder (string) – Folder in which the data spectra_MICA_LAB_info.csv is stored. Default is pyFRESCO/data.

both_fit()[source]

Function to plot the fit of each single distribution, the global fit and the MSE per wavelength of the machine learning MGM.

Parameters:

None

Return type:

None

compare2(p, n_spectra, n_err, spectra, mineralname, smooth=False, save=False, fold=None, name=None)[source]

Function to show the comparison using the inferred absorption lines found using SpectraAnalysis.compare_lines(). This comparison will show three plots, one showing the target and neutral spectrum with the respective errors, one showing the (non)smooth ratioed spectrum and the MICA files CRISM spectrum with the inferred absorption lines and the last one being the laboratory spectrum with the inferred absorption lines.

Parameters:

  • p (float) – Separator for the double plot of the target and MICA spectra.

  • mineralname (string) – Name of the mineral as given in the Mineral Name column in the ‘spectra_MICA_LAB_info.csv’ file.

  • smooth (bool) – If True it uses the last smoothed spectrum.

  • save (string) – If to save the plot or not. Default is False.

  • fold (string) – If save is True this is the path in which to save the plot. If None then the home directory is selected. Default is None.

  • name (string) – Name of the plot if required to save it. Default is None.

Return type:

None

compare_lines(name, smooth=False, ran=10, s=5, t=10)[source]

Function that automatically infers the nearest absorption lines that are present within a set range from the tabulated absorption line position of the given mineral guess from the MICA siles CRISM reference spectra. It works by firstly search for local minima in the set interval, if it does not find any sufficiently good local minima it searches for inflection points using the KneeLocator function of the kneed module (https://pypi.org/project/kneed/).

Parameters:

  • name (string) – Name of the mineral as given in the Mineral Name column in the ‘spectra_MICA_LAB_info.csv’ file.

  • smooth (bool) – If True it uses the last smoothed spectrum.

  • ran (float) – Interval semi-width for the check of elbow points. Deafult is 10.

  • s (float) – Sensibility of the elbow finding function. Default is 5.

  • t (float) – Threshold limit value for proximity with tabulated absorption. Default is 10.

Returns:

  • absoC (list) – Position of the inferred lines from the comparison with MICA CRISM reference spectra.

  • absoL (list) – Position of the inferred lines from the comparison with the Laboratory reference spectra.

  • diffC (list) – Difference between the inferred lines and the MICA CRISM reference absorption lines.

  • diffL (list) – Difference between the inferred lines and the Laboratory reference absorption lines.

  • CRISM_abs (list) – MICA CRISM reference absorption lines.

  • LAB_abs (list) – LAboratory reference absorption lines.

compare_spectra(mineralname, smooth=False, errorplot=False, use='LAB', alpha=0.15, folder=None, size=[3, 15], save=False, namesave=None, folder_save=None, ext='.png', show=True)[source]

Function to do the spectra comparing using both the MICA spectra and the laboratory spectra. IMPORTANT: to use this function one must have the spectra_MICA_LAB_info.csv file and all the files it points to in the same folder!.

Parameters:

  • mineralname (string) – Name of the mineral as given in the Mineral Name column in the ‘spectra_MICA_LAB_info.csv’ file.

  • smooth (bool) – If True it uses the last smoothed spectrum.

  • errorplot (bool) – If to plot or not the errorbar fo the normalized spectrum. Default is False.

  • use (string {'MICA' , 'LAB' , 'Both'}) – Which reference to use for the comparison. For the first it uses the MICA spectra fro comaprison, for ‘LAB’ it uses the laboratory spectra and for ‘Both’ it uses both.

  • alpha (float) – Shade strength of the error plot if errorplot is True. Default is 0.15.

  • folder (string) – Folder in which the ‘spectra_MICA_LAB_info.csv’ is located. If None the folder is defaulted as the home fodler. Default is None.

  • size (list with two float values) – Size of the plot.

  • save (bool) – To save or not the plot.

  • namesave (string) – The name with thich you want to save the plot, if save is True a name must be given.

  • folder_save (string) – The folder path in which you want to save the plot, if None it will save the plot in the home folder.

  • ext (string {'.png' , '.jpg' , '.pdf'}) – Extension of the image.

  • show (bool) – To show or not the plot.

Return type:

None

convex_hull(interp='linear', plot=False)[source]

Function to apply the convex hull caseline correction before feeding a spectrum to the mgm.

Parameters:

  • interp (string)

  • plot (bool) – To plot or not the result. Default is False.

Returns:

final – baseline-corrected spectrum. WARNING: THIS FUNCTION RE-INITIALIZE THE self.final SPECTRUM!

Return type:

numpy 1-d array

create_animation(losses, gaussian_sums, data, n_wav, frame_step=10, FPS=15, save=False, path=None, name='mgm_gradient_descent')[source]

Function to create animation of the fitting process for the SpectraAnalysis.mgm() function.

Parameters:

  • losses (list or array) – Losses per epoch.

  • gaussian_sums (2-dim array) – MGM fit per epoch.

  • data (list or array) – Normlized/smoothed spectrum.

  • n_wav (list or array) – Normalized wavelengths.

  • frame_step (int) – Frame step for the animation. Default is 20.

  • FPS (int) – Photograms frequency for the animation. Default is 15.

  • save (bool) – If to save or not the animation. Default is False.

  • path (string) – Path to which to save the animation. If None then the home directory is used. Default is None.

  • name (string) – Name of the animation if wanted to be saved. Default is mgm_gradient_descent.

Return type:

None

dataset_interaction(mineralname, use)[source]

Utils function to interact with the .csv file named “spectra_MICA_LAB_info.csv” to extract the needed spectral informations from the comparison with the references given by the MICA files, 2019 (http://crism.jhuapl.edu/data/mica/). IMPORTANT: THE MICA FILES REFERENCE LABORATORY SPECTRA ARE IN https://crismtypespectra.rsl.wustl.edu/ AND FOR SOME REASONS SOME SPECTRA RESULTS UNAVALIABLE FROM THERE, BEING: CO2 ICE, HYDRATED SILICA, HYDROXYLATED FE SULFATE, GYPSUM AND CHLORIDE. THUS, REGARDING THESE MINERALS ONLY THE CRISM REFERENCE IS USED AT THE MOMENT.

Parameters:

  • mineralname (string) – Name of the mineral from the MICA files.

  • use (string) – If to use only the CRISM certified spectra, the laboratory spectra or both of them. Accepted values are either MICA, LAB or Both.

Returns:

index – Index of the given mineral name in the spectra_MICA_LAB_info.csv .

Return type:

int

find_nearest(array, value)[source]

Function used to find the index of element nearest to a given arbitrary value.

Parameters:

  • array (array) – Array in which to search

  • value (float) – Value to search the nearest element inside array.

Returns:

idx – Index of the nearest value.

Return type:

int

global_fit()[source]

Function to plot the fit of the machine learning MGM with the resulting mean square error per wavelength on top.

Parameters:

None

Return type:

None

limits(other_w=None)[source]

Function to compute the limits of the x-axis and the corresponding indexes to then compute the y-axis limits of an array given the list/array of the x axis.

Parameters:

other_w (list, array or None) – If list or array, this will be taken as the wavelength array to use, if None then the CRISM observation wavelengths are used. Default is None.

Returns:

extremas – list of index correspondant do xmin , index correpsondant to xmax, xmin and xmax

Return type:

list of float

local_fit()[source]

Function to plot the fit of each single distribution of the machine learning MGM.

Parameters:

None

Return type:

None

mgm(n, iterations, lr, means=None, mean_ranges=[], asym=False, smooth=False, hull=False, interp='linear', t='savgol', w=0, o=0)[source]

Machine learning driven Modified Gaussian Model (MGM, https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/JB095iB05p06955?src=getftr). This version of the algorithm implements gradient-descent, thus it efficiently fits the reference spectrum with the required number of skew normal distributions (https://en.wikipedia.org/wiki/Skew_normal_distribution) formulated as follows:

\[G(\lambda;\alpha , \mu , A , f) = A\cdot e^{-4ln(2)\left(\frac{\lambda - \mu}{f}\right)^{2}}\cdot\left[1 + erf\left(2\sqrt{ln(2)}\alpha\frac{\lambda - \mu}{f}\right)\right]\]

Where λ is the array of wavelengths, α is the skewing parameter, μ is the mean (i.e. the line position), A is the amplitude and f is the full width at half maximum. The used loss function is Pytorch.F Mean Square Loss (MSE, https://pytorch.org/docs/stable/generated/torch.nn.functional.mse_loss.html).

Parameters:

  • n (int) – Number of skew normal distributions to be fitted.

  • max_iterations (int) – Number of iterations.

  • lr (float) – Learning rate for the Adam orptimizer.

  • means (list of float or None) – If None, then the means (μ) are inserted in the fitting parameters. If a list is given, then the values in the list will be used as mean values for the distributions. Default is None.

  • mean_ranges (list of tuples of float or None) – If None, no ranges is given to the distributions for the fit. If the list of tuples is given, then the search of the mean will be constrained inside an interval (min,max).

  • asym (bool) – If True, then the distributions fitted are Skew Normal Distributions, if False, then the alpha parameters will be set to zero, thus efficiently modeling the absorptions as simple Gaussian distributions. Default is False.

  • smooth (bool) – Whether or not to smooth the spectrum before the run. Default is False.

  • hull (bool) – Whether or not to correct by baseline the spectrum before the run. This can be useful when working with spectra not obtained thourgh the convex hull normalization method, sine their values can be higher than one, exceeding so the maximum value fot this function. Default is False

  • interp (string) – Type of interpolation, can be ‘linear’, ‘nearest’, ‘nearest-up’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’, or ‘next’. Default is ‘linear’.

  • t (string) – Type of smoothing algorithm to be used. Must be either ‘savgol’ (Savitzky-Golay filter) or ‘movmean’ (Moving Average filter). Default is ‘savgol’.

  • w (int) – Smoothing window size. Default is 0.

  • o (int) – Polynomial order for the Savitzky-Golay smoothing filter, can be between 1 and 5. Default is 0.

Returns:

  • losses (array) – Array of the loss value for each epoch.

  • gaussian_sums (array) – Resulting fit per epoch.

  • n_wav (array) – Normalized wavelength array.

  • gaussian_sum (array) – Final fit.

  • a (list) – Amplitude (A) parameters of the distributions.

  • m (list) – Mean (μ) parameters of the distributions.

  • fwhm (list) – Full width at half maximum parameters (f) of the distributions.

  • alpha (list) – Skew parameters (α) of the distributions.

moving_average(window_size, limiti=True)[source]

Function to perform a moving average smoothing on the normalized spectrum.

Parameters:

window_size (int) – Size of the step taken for the moving mean.

Returns:

result – Moving-mean smoothed normalized spectrum.

Return type:

1-dim array

mse()[source]

Calculates the mean square error:

\[mse = (y_{fit} - y)^{2}\]
Parameters:

None

Returns:

residuals – Mean square error of the fit per wavelength.

Return type:

array

savgol(window, order, limiti=True)[source]

Function to perform a Savitzky-Golay smoothing on the normalized spectrum as given in https://pubs.acs.org/doi/10.1021/ac60214a047. Mutuated from scipy.signal (https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.savgol_filter.html).

Parameters:

  • window (int) – Size of the convolution window.

  • order (int) – Order of the convolutional fit.

Returns:

result – Moving-mean smoothed normalized spectrum.

Return type:

1-dim array

simple_compare(mineralname, smooth=False, use='LAB', alpha=0.15, folder=None, errorplot=False)[source]

Function to do a simple spectra compariSON using both the MICA spectra and the laboratory spectra. To do so it simply plot the smoothed or non-smoothed normalized spectrum and it superimposes the absorption lines of the mineral guess given in the input. IMPORTANT: to use this function one must have the spectra_MICA_LAB_info.csv file and all the files it points to in the same folder!.

Parameters:

  • mineralname (string, case sensitive) – Name of the mineral as given in the Mineral Name column in the spectra_MICA_LAB_info.csv file.

  • smooth (bool) – If True it uses the last smoothed spectrum.

  • use (string {'MICA' , 'LAB' , 'Both'}) – Which reference to use for the comparison. For the first it uses the MICA spectra fro comaprison, for ‘LAB’ it uses the laboratory spectra and for ‘Both’ it uses both.

  • alpha (float) – Shade strength of the error plot if errorplot is True. Default is 0.15.

  • folder (string) – Folder in which the ‘spectra_MICA_LAB_info.csv’ is located. If None the folder is defaulted as the home fodler. Default is None.

  • errorplot (bool) – If to plot or not the errorbar fo the normalized spectrum. Default is False.

Return type:

None

upload_norm(name, folder)[source]

Function to upload a pre-made median/mad spectrum.

Parameters:

  • name (string) – Name of the file.

  • folder (string or None) – If None it will be saved into the home folder, if a folder path is given, the path must end with the /.

Returns:

  • norm (1d-array) – The uploaded median spectrum.

  • w (1d-array) – The cut wavelength range.