No Bulk Import for Maven Feed

[andrew_5903]

Hi
I am trying to set the drop path to do a bulk import with a maven feed. I create a new maven feed but can not find the option to set the bulk import drop path?

I am running ProGet Free Edition Version 5.3.17 (Build 19)

Thanks

[atripp]

Hello;

As of ProGet 5.3.17, ProGet does not support drop paths for Maven; the precise reason is complex, but it's ultimately a result of the way Maven is structured (i.e. a file system vs packages), and how files can be added piece-meal.

It is something we are considering adding in a future release, but it's complex. Other users have reported that they've used a process like this for a disk-based repository.

1. Traverse all directories and upload all POM files with a path relative to the root
2. Traverse all directories again, and upload all non-POM and non-checksum files (like .md5)

There will be errors, particularly if you have invalid POM files or your directory structure doesn't match the required MAVEN convention, so inspect those case-by-case to determine if it matters (like an bad artifact from 5 years ago can probably be ignored).

Unfortunately we don't have a lot more details, but we'd love to hear more from your experience if you end up scripting this,

Cheers,
Alana

[andrew_5903]

Hi @atripp

Thanks for your response and letting me know that the functionality is not available anymore I thought i was going mad.

So is the only way to bulk upload artefacts to ProGet based on the steps that you just outlined?

If I physically copied the files across into the feed location directory in the directories based on its package names that would be sufficient? Do I not need ProGet to re-index the feed?

Do you know of any other way to do this? Possibly using maven maybe?

If I installed a legacy version of ProGet to get the functionality back is that an option I could take too?

Thanks for your help
Andrew

[andrew_5903]

@atripp

I tried adding a new artefact to file system in the stored location defined by my maven feed, but I cant get the feed to pickup the new artefact. I have tried the following to get it to pick it up

1. Feed re-index: Manage Feed > Storage & Retention > Page Storage > Re-index
2. Trigger index task: Admin > Scheduled Jobs > BuildMavenFeedIndex (for feed) > Run
3. Alternate task: Admin > Manage Service > Schedule Job Dispatcher > Run

After running all of these my feed still does not pick up my artefact that I just added. Any ideas how I get the new artefact to be index in the database?

Thanks

[atripp]

Hi Andrew,

The only way to get a Maven artifact in ProGet is by using the API to push it. The re-indexing can detect missing files on disk, but it does not add new files.

This is by design, we simply never created this functionality for ProGet. Even in old versions.

We may add it later, but there's not a lot of demand, since it's "relatively easy" to write a script to import everything in via the API. Really it's just doing a number of HTTP Push commands by starting wtih the POM files as i mentioned.

Cheers,
Alana

[andrew_5903]

Thanks @atripp

I have ended up achieving this using the maven plugin deploy-file and pushing the pom and jar into the API endpoint

For anyone elses reference if they are interested:

mvn deploy:deploy-file -Durl=<api url for feed> -Dfile=<jar> -DpomFile=<pom>

Thanks for your help

[atripp]

@andrew_5903 thanks, I'd like to update the docs! Did you end up writing a script to just call that for each Jar+Pom in your directory?