Package loci.common

Class S3Handle

All Implemented Interfaces:
DataInput, DataOutput, IRandomAccess

public class S3Handle extends StreamHandle
Provides random access to S3 buckets using the IRandomAccess interface. Instances of S3Handle are read-only.
See Also:
  • Field Details

    • DEFAULT_S3_PROTOCOL

      public static final String DEFAULT_S3_PROTOCOL
      Default protocol for fetching s3://
      See Also:
    • SCHEME_PARSER

      protected static final Pattern SCHEME_PARSER
    • S3_MAX_FORWARD_SEEK

      protected static final int S3_MAX_FORWARD_SEEK
      If seeking more than this distance reset and reopen at offset
      See Also:
  • Constructor Details

    • S3Handle

      public S3Handle(String url) throws IOException
      Open an S3 file
      Parameters:
      url - the full URL to the S3 resource
      Throws:
      IOException - if there is an error during opening
    • S3Handle

      public S3Handle(String uristr, boolean initialize, StreamHandle.Settings s) throws IOException
      Open an S3 file
      Parameters:
      uristr - the full URL to the S3 resource
      initialize - If true open the stream, otherwise just parse connection string
      s - custom settings object
      Throws:
      IOException - if there is an error during opening
  • Method Details

    • canHandleScheme

      public static boolean canHandleScheme(String url)
      Return true if this is a URL with an s3 scheme
      Parameters:
      url - URL
      Returns:
      true if this class can handle url
    • connect

      protected void connect() throws IOException
      Connect to the server
      Throws:
      IOException - if there was an error connecting to the server
    • initialize

      protected void initialize() throws IOException, S3ClientServiceException
      Check bucket or object exists
      Throws:
      IOException - if unable to get the object
      S3ClientServiceException - if unable to get the object
    • getServer

      public String getServer()
    • getPort

      public int getPort()
    • getBucket

      public String getBucket()
    • getPath

      public String getPath()
    • cacheObject

      public static String cacheObject(String url, StreamHandle.Settings s) throws IOException, HandleException
      Download an S3 object to a file system cache if it doesn't already exist
      Parameters:
      url - the full URL to the S3 resource
      s - custom settings object
      Returns:
      File path to the cached object
      Throws:
      IOException - if there is an error during reading or writing
      HandleException - if no destination for the cache is provided
    • getCacheKey

      public String getCacheKey()
    • downloadObject

      protected void downloadObject(Path destination) throws HandleException, IOException
      Throws:
      HandleException
      IOException
    • isBucket

      public boolean isBucket()
      Is this an accessible bucket? TODO: If this bucket doesn't exist do we return false or thrown an exception?
      Returns:
      True if a bucket
    • length

      public long length() throws IOException
      Description copied from interface: IRandomAccess
      Returns the length of this stream.
      Specified by:
      length in interface IRandomAccess
      Overrides:
      length in class StreamHandle
      Returns:
      the length in bytes of the stream
      Throws:
      IOException - if the length cannot be retrieved
    • seek

      public void seek(long pos) throws IOException
      Description copied from interface: IRandomAccess
      Sets the stream pointer offset, measured from the beginning of this stream, at which the next read or write occurs.
      Specified by:
      seek in interface IRandomAccess
      Overrides:
      seek in class StreamHandle
      Parameters:
      pos - new byte offset (pointer) in the current stream. Unless otherwise noted, may be larger or smaller than the current pointer, but must be non-negative and less than the value of #length()
      Throws:
      IOException - if pos is invalid or the seek fails
      See Also:
    • resetStream

      protected void resetStream() throws IOException
      Description copied from class: StreamHandle
      Close and reopen the stream; the stream pointer and mark should be reset to 0. This method is called if we need to seek backwards within the stream.
      Specified by:
      resetStream in class StreamHandle
      Throws:
      IOException - if the stream cannot be reset
      See Also:
    • exists

      public boolean exists() throws IOException
      Does this represent an accessible location?
      Specified by:
      exists in interface IRandomAccess
      Overrides:
      exists in class StreamHandle
      Returns:
      true if this location is accessible
      Throws:
      IOException - if unable to determine whether this location is accessible
    • resetStream

      protected void resetStream(long offset) throws IOException
      Reset the stream to an offset position
      Parameters:
      offset - Offset into object
      Throws:
      IOException - if there is an error during reading or writing
    • toString

      public String toString()
      Overrides:
      toString in class Object