Hello; I've updated the documentation to clarify this, but it's available starting in ProGet 5.2.9. So, you'll need to upgrade to enable it :)
atripp
@atripp
C# developer by trade, but writing less and less code and more and more specs.
Best posts made by atripp
-
RE: Service Health API call returning 404
-
RE: NPM Connector returns plus "+" in versions
Thanks for the update! I've noted this in the docs, and linked to this discussion :)
https://github.com/Inedo/inedo-docs/commit/d24087911584bbda833314084a58c2ae1ff41c39
-
RE: [ProGet] [NativeApi] NpmPackages_DeletePackage not working.
Hello,
That API will only delete package metadata from the database, not from disk. It's mostly intended for internal use only, and probably shouldn't be exposed to the API. In any case, we don't store the
@
with internally, so if you change@myscope
tomyscope
it should work.Note that the NPM doesn't provide a way to delete packages, and we never implemented it. There hasn't been any demand for it to date, as people don't really delete packages programmatically - but you're definitely welcome to submit a feature request and help us understand why it'd be a value (like, the workflow you use that requires deleting packages, etc).
Alana
-
RE: Creating PowerShell repository, protecting pull/download by API key
Hello, for sure!
It's pretty easy; just don't give the
Anonymous
user any access to your feeds, and then authentication will always be required, either when browsing the ProGet application or using the API (such asInstall-Module
).When you use the Register-PSRepository command, you can the
Credential
option to specify a credential.This credential can be the name/password of a user inside of ProGet (let's say,
Admin:Admin
), or it can be username ofapi
with a password of an api key you've configured (so,api:my-secret-key
). -
RE: Restricting API access to View/Download
Hello;
The Native API is for low, system-level functions, and it's "all or nothing". If you give someone access to Native API, you are effectively making them an administrator, as they can also change permissions and grant admin privileges. So, I don't think you want this. Instead, you'll want to use the Debian API endpoint that we implement.
It's a third-party API format
In order to support third-party package formats types like NuGet, npm, etc., ProGet implements a variety of third-party APIs. We only provide minimal documentation for these APIs, as they are generally either already documented elsewhere. However, you can generally find the basics by searching for specific things you'd like to do with the API, such as "how to search for packages using the NuGet API" or "how to publish an npm package using the API".
So in this case, I recommend to search "how to view and download apt packages".
-
RE: PyPI package not shown in search results accessible via url
I'm not very familiar with PyPi packages, but I know there are some oddities with
-
and_
, and that they are sometimes supposed to be treated the same, and sometimes not. We don't totally understand all the rules, to be honest (even after reading PEP503 specifications).In this case, the package is actually
websocket_client
, notwebsocket-client
.See: https://pypi.org/project/websocket_client/
When you search for
websocket_client
in ProGet, it shows up, as expected. -
RE: How to find out package disk space?
In ProGet 5.3, we plan to have a couple tabs on each
Tag
(i.e. container image) that would provide this info: Metadata (will be a key/value pair of a bunch of stuff), andLayers
will show details about each of these layers.That might help, but otherwise, we have retention policies which are designed to clean up old and unused images.We'll also have a way to detect which images are actually being used :)
-
RE: [BUG - ProGet] Not able to remove container description
As @apxltd mentioned, we've got a whole bunch planned for ProGet 5.3.
I've logged this to our internal project document, and if it's easy to implement in ProGet 5.2 (I can't imagine it wouldn't be), we'll log it as a bug and ship in a maintence release.
Do note, this is not an IMAGE description, it's a REPOSITORY (i.e. a collection of images with the same name, like
MyCoolContainerApp
) description; so this means the description will be there on all images/tags in the repository. -
RE: [Question - ProGet] Are versions amount wrong ?
You're right, I guess that's showing the "layers" instead of the "tags"; I think it should be showing container registries separately (they're not really feeds), but that's how it's represented behind the scenes now.
Anyways we are working on ProGet 5.3 now; there's a whole bunch of container improvements coming, so I've noted this on our internal project document, to make sure we get a better display for container registries.
-
RE: Anonymous user can see list of packages and containers
@Stephen-Schaff thanks for the bug report, I verified that this may happen depending on permission of user, and which feeds they can/can't use --- but it seems an easy enough fix that we can do via PG-1894 (targeted to next release) - the packages can't be viewed upon clicking, but it's a sub-optimal experience for showing packages they can't see
Latest posts made by atripp
-
RE: GPG error updating Debian repositories
@daniel-scati great work finding that :)
The Feed Description is written in the Release Index, and the Debian "Stanza" format is kind of weird like that.
However, newlines should already be replaced...
var desc = FeedCache.GetFeed(this.feedId)?.Feed_Description; if (!string.IsNullOrWhiteSpace(desc)) writer.WriteLine($"Description: {desc.ReplaceLineEndings("\n ")}");
A blank line with a space isn't supposed to create a new "stanza", but maybe it is in your version?? We'll look closer; perhaps we should just strip the newline altogether and replace it with a space.
Cheers,
Alana -
RE: Errors in PackageAnalyzer logs
Thanks for clarifying, that's good to know taht solved the problem! I looked again and found it. Maybe I was looking in the wrong database
It doesn't look like bootstrap.less is in your feeds, but it is in the
PackageNameIds
table:
Just because it's in that table doesn't mean you're using it; we use the PackageName_Id as a surrogate ID instead of the package name. There should only be one entry, since NuGet package names are case insensitive.
anyway, I think I found the culprit -- looks like it was added during a vulnerability database update. MAybe there is some bug that didn't properly check the casing
Anyways, that's what we have to clean-up up :)
-
RE: Errors in PackageAnalyzer logs
Hi @v-makkenze_6348 ,
Just to give you an update, we're still working on this; there's a lot of duplicated data and figuring out how to clean it up is a challenge. It's hard to say where it came from.
For example, your database has internal ids for both
bootstrap.less-3.4.1
andBootstrap.Less-3.4.1
, and we need to figure out how safely merge those. At this point, we're thinking we're just going to do one massive SQL Script to do a cleanup. Then, we can try to figure out where the duplicates are coming from.Regarding
VicreaNeuronGegevensmagazijnContracts
package, we can't find that in your database. But I assume that's a new package/version. If you have a newer backup, please upload that to the link we sent - we can check it in there. I think its' related to 4-part version, but just want to check.Cheers,
Alana -
RE: ProGet: AD/LDAP issues with 2024 Docker image
Hi @scott-goodwin_8239 ,
Thanks for the detailed report; you're right, it's definitely related to some kind of library/platform problem. There seems to be a known bug in one of the libraries that can cause this in some cases on Linux.
Fortunately we already identified a good workaround/fix to this issue (PG-2626), and we plan to ship it on Friday evening; please let us know if you'd like to try this in a prerelease version.
Thanks,
Alana -
RE: Conda Feed to SMB Share
Hi @scott-wright_8356 ,
Thanks; it's hard to say what the issue is, and unfortunately we don't really have any good troubleshooting tools here. That error is coming from within SQL Lite, and we can't reproduce it when we do a SMB share.
I wonder if you can follow some of the same troubleshooting steps? Specifcally like using ProcMon or mounting the network drive with SysInternals?
We just don't really have any means to troubleshoot this, so we'd need to go back to the drawing board to figure out why it doesn't work for you. You're welcome to keep exploring C#, but let me share the code that's closer to ProGet.
Here is the code that ProGet is running. It's kind of similar, but a little different. Note we are using
System.Data.SQLite.Core-1.0.118
and .NET6.private static SQLiteConnection OpenOrCreateDatabase(string fileName, out Dictionary<string, string> config) { var str = new SQLiteConnectionStringBuilder { DataSource = fileName, FailIfMissing = false }.ToString(); if (FileEx.Exists(fileName)) { var conn = new SQLiteConnection(str); try { conn.Open(); config = ReadConfig(conn); if (config.TryGetValue("SchemaVersion", out var versionStr)) { int version = int.Parse(versionStr); if (version == 1) { using var cmd2 = new SQLiteCommand(GetScript("UpdateSchemaToV2"), conn); cmd2.ExecuteNonQuery(); return conn; } else if (version == 2) { return conn; } } } catch { } conn?.Dispose(); FileEx.Delete(fileName); } else { DirectoryEx.Create(PathEx.GetDirectoryName(fileName)); } var conn2 = new SQLiteConnection(str); conn2.Open(); using var cmd = new SQLiteCommand(GetScript("CreateSchema"), conn2); cmd.ExecuteNonQuery(); cmd.CommandText = GetScript("UpdateSchemaToV2"); cmd.ExecuteNonQuery(); config = ReadConfig(conn2); return conn2; } private static Dictionary<string, string> ReadConfig(SQLiteConnection conn) { using var cmd = new SQLiteCommand("SELECT Key_Name, Value_Text FROM OtherData", conn); using var reader = cmd.ExecuteReader(); var res = new Dictionary<string, string>(); while (reader.Read()) { var key = reader.GetString(0); var value = reader.IsDBNull(1) ? null : reader.GetString(1); res[key] = value; } return res; }
Here is the
CreateSchema
script:CREATE TABLE OtherData ( Key_Name TEXT PRIMARY KEY NOT NULL, Value_Text TEXT ); INSERT INTO OtherData (Key_Name, Value_Text) VALUES ('SchemaVersion', '1'); CREATE TABLE FetchInfo ( Subdir_Name TEXT PRIMARY KEY NOT NULL, Modified_Date INTEGER, Fetch_Date INTEGER NOT NULL ); CREATE TABLE ChannelData ( Package_Name TEXT PRIMARY KEY NOT NULL, Flags_Value INTEGER NOT NULL, Description_Text TEXT, DevUrl_Text TEXT, DocSourceUrl_Text TEXT, DocUrl_Text TEXT, Home_Text TEXT, Icon_Hash TEXT, Icon_Url TEXT, License_Text TEXT, RunExports_Json BLOB, SourceGitUrl_Text TEXT, SourceUrl_Text TEXT, Summary_Text TEXT, Timestamp_Value INTEGER, Version_Text TEXT NOT NULL, Subdirs_Csv TEXT NOT NULL ); CREATE TABLE RepoData ( Package_Name TEXT NOT NULL, Subdir_Name TEXT NOT NULL, Version_Text TEXT NOT NULL, Build_Text TEXT NOT NULL, ArchiveType_Code INTEGER NOT NULL, Build_Number INTEGER NOT NULL, License_Text TEXT, LicenseFamily_Text TEXT, MD5_Hash BLOB, SHA256_Hash BLOB, Package_Size INTEGER NOT NULL, Timestamp_Value INTEGER, Dependencies_Json BLOB, AppEntry_Text TEXT, AppType_Text TEXT, Type_Text TEXT, Summary_Text TEXT, Icon_Hash BLOB, PRIMARY KEY (Package_Name, Version_Text, Subdir_Name, Build_Text, ArchiveType_Code) );
Here is the
UpdateSchemaToV2
script:UPDATE OtherData SET Value_Text = '2' WHERE Key_Name = 'SchemaVersion'; ALTER TABLE RepoData ADD Constraints_Json BLOB;
From there, it's just a bunch of inserts into those tables. The "unable to open database file" error would probably be occurring in the middle of those, but it's hard to say.
The race condition occurs when multiple threads run
OpenOrCreateDatabase
at same time. It's rare as I mentioned.Alana
-
RE: [BM / OT] Renaming "user/password" or "private key" credentials breaks Linux config
This is by design, as a common use case is to delete/create a credential with the same name. The design pattern is based around things like Windows Services, and the fact that renaming a username will not update all Windows services, etc.
However, it's really inconvenient, and it'd be nice if there was a " would you like to also rename things in scripts/resources/servers/etc". But there's not, and it's something on our "to consider" list for an upcoming version.
It's easy enough to work-around though - and as an FYI, in BuildMaster, you can just export the infrastructure, then reimport it after search/replacing the credential name in the JSON file.
Cheers,
Alana -
RE: Warnings about a package in Diagostic Center without package name
Hi @v-makkenze_6348 ,
Thanks for the report; please ignore this, it should not be logged in the Diagnostic Center. It's addressed via PG-2625 in ProGet 2024.1
Cheers,
Alana -
RE: Conda Feed to SMB Share
Hi @scott-wright_8356 ,
ProGet doesn't "directly" work with SMB shares, but instead uses the File System API. As such there's no special configuration or ports required - that's all handled at the operating system level. And it sounds like it's working fine, since other feeds are working.
As for SQL Lite, a few feed types (including conda, debian, rpm, etc.) will use a SQL Lite to "cache" connector information. But a SQL Lite database is just a file stored on disk, as you've noticed - if ProGet can read/write files on an SMB share, there is no reason it wouldn't work.
We haven't had issues in HA configurations with SQL Lite connectors, but we have seen this issue come twice before. One time was a ticket, and it was some "malware" that was blocking the SQL Lite file for security reasons. A more recent thread is here: https://forums.inedo.com/topic/4128
I wonder if you can follow some of the same troubleshooting steps? In the end, this seemed to work:
I was finally able to try this with a mounted network drive (using SysInternals), and that works
We don't really have an alternate solution at this time, but if this continues to be an issue we can definitely consider a node-by-node / local file system configuration.
Thanks,
Alana -
RE: Updating to Proget to 24.0.0 from 23.0.18 gives "Cannot drop the view 'database_firewall_rules', because it does not exist or you do not have permission."
Thanks for confirming that @carl-westman_8110 ! That should definitely not be showing up :)
This is definitely a bug on Microsoft's end (we've seen similar in some early distributions of SQL Server 2008). We will work around this in 2024.1 via PG-2624 - we are planning to ship on Friday.
As an FYI, here is the new script:
DECLARE @SQL NVARCHAR(MAX) SET @SQL = '' SELECT @SQL = @SQL + 'DROP PROCEDURE ' + QUOTENAME(name) + ' ' FROM sys.procedures WHERE schema_id <> 4 SELECT @SQL = @SQL + 'DROP TYPE ' + QUOTENAME(name) + ' ' FROM sys.table_types WHERE schema_id <> 4 SELECT @SQL = @SQL + 'DROP VIEW ' + QUOTENAME(name) + ' ' FROM sys.views WHERE schema_id <> 4 SELECT @SQL = @SQL + 'DROP FUNCTION ' + QUOTENAME(name) + ' ' FROM sys.objects WHERE type IN ('FN','TF') and schema_id <> 4 EXEC sp_executesql @SQL
In the likely event that Azure SQL slips another sys object into those tables, this will at least ignore that error.
-
RE: GPG error updating Debian repositories
@philippe-camelio_3885 thank you much, we just updated the docs :)