Quantcast

Pulsar Metadata

classic Classic list List threaded Threaded
2 messages Options
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Pulsar Metadata

Joe Greer

 

Hi All,

 

We’ve been using Pulsar to send jobs to our cluster on campus. Generally we send a dataset list of files over. We’re wondering if there is a way to get the metadata for the files after they’ve been sent to the cluster. We’re specifically trying to find the filepaths of the files before they were through Pulsar to the cluster.

 

Any guidance would be great.

 

Thanks,

 

Joe


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pulsar Metadata

John Chilton-4
I perhaps don't understand the question. As long as the "datasets" are
Galaxy datasets and they appear in the command-line specific by the
"command" block of your tool - it should be transmitted. When you say
a "dataset list of files" - do you mean (1) a Galaxy collection, (2) a
set of files selected with a <param type="data" multiple="true" />
block in your tool, or (3) do you have a special datatype that
references lists of paths on the file system not tracked by Galaxy?

Pulsar doesn't really have an API to expose these things to your tools
- as long as you are sticking to Galaxy's abstractions these things
should just work. If you have tools or datatypes that reference
arbitrary other files on the file system (like 3 above) that is going
to break some Galaxy abstractions including Pulsar, quota tracking,
purging datasets, object stores, etc....

-John

On Thu, Feb 2, 2017 at 8:49 AM, Joseph Brent Greer
<[hidden email]> wrote:

>
>
> Hi All,
>
>
>
> We’ve been using Pulsar to send jobs to our cluster on campus. Generally we
> send a dataset list of files over. We’re wondering if there is a way to get
> the metadata for the files after they’ve been sent to the cluster. We’re
> specifically trying to find the filepaths of the files before they were
> through Pulsar to the cluster.
>
>
>
> Any guidance would be great.
>
>
>
> Thanks,
>
>
>
> Joe
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Loading...