gl.projects.get() fails when provided "namespace/project_name"

Description of the problem, including code/CLI snippet

In project-status.py ...

    for this_project in options.project:
        this_project = my_gitlab.projects.get(this_project)
        print("Adding project:", this_project.path_with_namespace)
        projects = projects + [this_project]

when this_project provided in "namespace/project_name" format, projects.get() fails with 404 error.
when this_project specified in project_id format, projects.get() succeeds.
same API access token for both forms of request.
access token is a project level token with full rights.

Expected Behavior

this_project = my_gitlab.projects.get("namespace/project_name") succeeds

Actual Behavior

this_project = my_gitlab.projects.get("namespace/project_name") fails with 404 error

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 330, in wrapped_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/gitlab/mixins.py", line 139, in get
    server_data = self.gitlab.http_get(path, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 790, in http_get
    result = self.http_request(
  File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 756, in http_request
    raise gitlab.exceptions.GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 404: 404 Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/wrowe/git-repos/python-gitlab/./project-status.py", line 139, in <module>
    this_project = my_gitlab.projects.get(this_project)
  File "/usr/local/lib/python3.10/site-packages/gitlab/v4/objects/projects.py", line 775, in get
    return cast(Project, super().get(id=id, lazy=lazy, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 332, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabGetError: 404: 404 Not Found

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894 
Adding project: https-phase2/ha-proxy-migration
Scanning issues for ha-proxy-migration ...
Building report for ha-proxy-migration ...
Writing report file ...

Specifications

python-gitlab version: 3.6.0
API version you are using (v3/v4): v4
Gitlab server version (or gitlab.com): 14.0.12 Community Edition

Don't know if this is also helpful.

% gitlab -c ~/.python-gitlab.cfg -g gitlab-https project-notification-settings get --project-id https-phase2/ha-proxy-migration
Impossible to get object (gitlab.v4.objects.notification_settings.NotificationSettingsManager.get() got multiple values for keyword argument 'id')

@walterrowe

Hi @walterrowe

I know that it does work to get by the path as I use that often. So this is odd.

Can you add a print statement so we see the name of the project?

    for this_project in options.project:
        print(f"Project: {this_project!r}")
        this_project = my_gitlab.projects.get(this_project)
        print("Adding project:", this_project.path_with_namespace)
        projects = projects + [this_project]

See the first line of output for the failure. The last command line arg is the project with name space.

See the first line of output for the failure. The last command line arg is the project with name space.

But can you print the value that you are sending to the get() method please. I would like to see. One thought I had is that could possibly be iterating over the string and thus the project trying to be retrieved could be the letter h. Printing the value will help figure that out.

Output showing value and type ...

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration

Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>)

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 330, in wrapped_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/gitlab/mixins.py", line 139, in get
    server_data = self.gitlab.http_get(path, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 790, in http_get
    result = self.http_request(
  File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 756, in http_request
    raise gitlab.exceptions.GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 404: 404 Not Found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/wrowe/git-repos/python-gitlab/./project-status.py", line 140, in <module>
    this_project = my_gitlab.projects.get(this_project)
  File "/usr/local/lib/python3.10/site-packages/gitlab/v4/objects/projects.py", line 775, in get
    return cast(Project, super().get(id=id, lazy=lazy, **kwargs))
  File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 332, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabGetError: 404: 404 Not Found

argparser code that processes the command line option for project.

group1.add_argument(
    "--project",
    dest="project",
    type=str,
    help="list of one or more project paths or ids",
    nargs="+",
    action="extend",
)

Looking in mixins.py I see that GetMixin calls utils.EncodedID.

python-gitlab/gitlab/mixins.py

Lines 130 to 131 in d5de4b1

    
           if isinstance(id, str): 
        
               id = utils.EncodedId(id)

utils.py EncodedID() has these lines:

python-gitlab/gitlab/utils.py

Lines 127 to 128 in d5de4b1

    
           if isinstance(value, str): 
        
               value = urllib.parse.quote(value, safe="")

in a terminal window I can type this and it works.

>>> import urllib.parse
>>> urllib.parse.quote("https-phase2/ha-proxy-migration",safe="")
'https-phase2%2Fha-proxy-migration'

>>> this_project="https-phase2/ha-proxy-migration"
>>> urllib.parse.quote(this_project,safe="")
'https-phase2%2Fha-proxy-migration'

Very odd 😕

You can add this to your program after getting the gl=gitlab.Gitlab() value.

gl.enable_debug()

This will output the data that is being sent to the server.

Looking in mixins.py I see that GetMixin calls utils.EncodedID.

python-gitlab/gitlab/mixins.py

Lines 130 to 131 in d5de4b1

if isinstance(id, str):

id = utils.EncodedId(id)

The quoting is working as I expected:

>>> import gitlab.utils as utils
>>> a = "https-phase2/ha-proxy-migration"
>>> utils.EncodedId(a)
'https-phase2%2Fha-proxy-migration'

with debug enabled ...

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration

Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>)

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my.gitlab.server:443
send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1\r\nHost: my.gitlab.server\r\nUser-Agent: python-gitlab/3.6.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nPRIVATE-TOKEN: my_private_token\r\nContent-type: application/json\r\n\r\n'
reply: 'HTTP/1.1 404 Not Found\r\n'
header: Server: nginx
header: Date: Wed, 06 Jul 2022 17:07:29 GMT
header: Content-Type: application/json
header: Content-Length: 25
header: Connection: keep-alive
header: Cache-Control: no-cache
header: Vary: Origin
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: X-Request-Id: 01G7A7P9ERX4PV59KBYQF1SZ17
header: X-Runtime: 0.008983
DEBUG:urllib3.connectionpool:https://my.gitlab.server:443 "GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1" 404 25

[ the rest of the traceback is the same as above ]

Also if you want to hop on the Gitter channel I could try to help in a more interactive way. https://gitter.im/python-gitlab/Lobby

Would love the interactive help but I can't connect to the gitter.im channel from this restricted network.

One thing I was reading was this:
https://docs.gitlab.com/ee/api/#namespaced-path-encoding

A project’s path isn’t necessarily the same as its name. A project’s path is found in the project’s URL or in the project’s settings, under General > Advanced > Change path.

But it looks like it is correct 😕

Can you try this?

this_project = my_gitlab.projects.get(3894 )
print(f"{this_project.path_with_namespace!r}")

Want to make sure there isn't some extra unseen characters in the path.

Another possibility is that Gitlab 14.0 has a bug, but I wouldn't think so.

with debug enabled ...

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration

Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>)

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my.gitlab.server:443
send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1\r\nHost: my.gitlab.server\r\nUser-Agent: python-gitlab/3.6.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nPRIVATE-TOKEN: my_private_token\r> ```

The path being sent to Gitlab looks correct to me.

send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration

The program does that.

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894

Looking for project: '3894' (type: <class 'str'>)

Adding project: https-phase2/ha-proxy-migration
Scanning issues for ha-proxy-migration ...
Building report for ha-proxy-migration ...
Writing report file ...

The program does that.

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894

Looking for project: '3894' (type: <class 'str'>)

Adding project: https-phase2/ha-proxy-migration
Scanning issues for ha-proxy-migration ...
Building report for ha-proxy-migration ...
Writing report file ...

I don't see quotes around the project name in "Adding project" can you do it with

print(f"{this_project.path_with_namespace!r}")

Yes.

% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894

Looking for project: '3894' (type: <class 'str'>)

Adding project: 'https-phase2/ha-proxy-migration'
Scanning issues for ha-proxy-migration ...
Building report for ha-proxy-migration ...
Writing report file ...

the python script snippet:

    for this_project in options.project:
        print(f"\nLooking for project: {this_project!r} (type: {type(this_project)})\n")
        this_project = my_gitlab.projects.get(this_project)
        print(f"Adding project: {this_project.path_with_namespace!r}")
        projects = projects + [this_project]

is it possible this is a bug in the 14.0.12 CE API? we are upgrading to 15.x over time. gitlab upgrades require iterating over each version unfortunately.

Sadly I don't know. It seems like python-gitlab is sending the correct path to the Gitlab server based on the debug output.

The only other shot in the dark I have is to also print the url-encoded ID form.

encoded_id = gitlab.utils.EncodedId(this_project.path_with_namespace)
print(f"{encoded_id!r}")

And compare to what trying to send.

is it possible this is a bug in the 14.0.12 CE API? we are upgrading to 15.x over time. gitlab upgrades require iterating over each version unfortunately.

Possible yes. I don't know though. I would think it isn't as it seems like a big bug. But not sure.

Also. Is this failing for all/other projects when doing a get using a path?

RESOLVED:

On this specific gitlab instance, I had to use https://my.gitlab.server/toplevel_entrypath for url in ~/.python-gitlab.cfg

This fails:

[gitlab-https]
url = https://my.gitlab.server/
private_token = my_private_token
api_version = 4

This works:

[gitlab-https]
url = https://my.gitlab.server/gitlab
private_token = my_private_token
api_version = 4

We don't have this "toplevel_entrypath" layer in our paths on our other gitlab instances. I need to talk to her gitlab admin about that.

Woo hoo!! Glad you figured it out. I was perplexed for sure.

Just curious. Does the web_url attribute show the correct path? Maybe next time I will ask to see that value too 🤓

Also confusing to me why the integer ID worked. Weird.

The integer ID probably worked because gitlab searches a different field in the database for an ID versus for a namespace/project_name.

and since you asked ...

Adding project: 'https-phase2/ha-proxy-migration' ('https://my.gitlab.server/gitlab/https-phase2/ha-proxy-migration')

definitely worth also asking for the web_url in the future.

The integer ID probably worked because gitlab searches a different field in the database for an ID versus for a namespace/project_name.

and since you asked ...
Adding project: 'https-phase2/ha-proxy-migration' ('https://my.gitlab.server/gitlab/https-phase2/ha-proxy-migration')
definitely worth also asking for the web_url in the future.

Thanks. I'll try to remember that!

Navigation Menu

Repository navigation

gl.projects.get() fails when provided "namespace/project_name" #2130

Description of the problem, including code/CLI snippet

Expected Behavior

Actual Behavior

Specifications

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if isinstance(value, str):
	value = urllib.parse.quote(value, safe="")

gl.projects.get() fails when provided "namespace/project_name" #2130

Description

Description of the problem, including code/CLI snippet

Expected Behavior

Actual Behavior

Specifications

Activity

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

walterrowe commented on Jul 6, 2022

JohnVillalovos commented on Jul 6, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions