Skip to content

Commit ca58008

Browse files
nejchJohnVillalovos
nejch
authored and
JohnVillalovos
committed
feat(cli): allow options from args and environment variables
BREAKING-CHANGE: The gitlab CLI will now accept CLI arguments and environment variables for its global options in addition to configuration file options. This may change behavior for some workflows such as running inside GitLab CI and with certain environment variables configured.
1 parent e19e4d7 commit ca58008

7 files changed

Lines changed: 421 additions & 25 deletions

File tree

‎docs/cli-usage.rst‎

Lines changed: 50 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,60 @@
33
####################
44

55
``python-gitlab`` provides a :command:`gitlab` command-line tool to interact
6-
with GitLab servers. It uses a configuration file to define how to connect to
7-
the servers. Without a configuration file, ``gitlab`` will default to
8-
https://gitlab.com and unauthenticated requests.
6+
with GitLab servers.
7+
8+
This is especially convenient for running quick ad-hoc commands locally, easily
9+
interacting with the API inside GitLab CI, or with more advanced shell scripting
10+
when integrating with other tooling.
911

1012
.. _cli_configuration:
1113

1214
Configuration
1315
=============
1416

15-
Files
16-
-----
17+
``gitlab`` allows setting configuration options via command-line arguments,
18+
environment variables, and configuration files.
19+
20+
For a complete list of global CLI options and their environment variable
21+
equivalents, see :doc:`/cli-objects`.
22+
23+
With no configuration provided, ``gitlab`` will default to unauthenticated
24+
requests against `GitLab.com <https://gitlab.com>`__.
25+
26+
With no configuration but running inside a GitLab CI job, it will default to
27+
authenticated requests using the current job token against the current instance
28+
(via ``CI_SERVER_URL`` and ``CI_JOB_TOKEN`` environment variables).
29+
30+
.. warning::
31+
Please note the job token has very limited permissions and can only be used
32+
with certain endpoints. You may need to provide a personal access token instead.
33+
34+
When you provide configuration, values are evaluated with the following precedence:
35+
36+
1. Explicitly provided CLI arguments,
37+
2. Environment variables,
38+
3. Configuration files:
39+
40+
a. explicitly defined config files:
41+
42+
i. via the ``--config-file`` CLI argument,
43+
ii. via the ``PYTHON_GITLAB_CFG`` environment variable,
44+
45+
b. user-specific config file,
46+
c. system-level config file,
47+
48+
4. Environment variables always present in CI (``CI_SERVER_URL``, ``CI_JOB_TOKEN``).
49+
50+
Additionally, authentication will take the following precedence
51+
when multiple options or environment variables are present:
52+
53+
1. Private token,
54+
2. OAuth token,
55+
3. CI job token.
56+
57+
58+
Configuration files
59+
-------------------
1760

1861
``gitlab`` looks up 3 configuration files by default:
1962

@@ -35,8 +78,8 @@ You can use a different configuration file with the ``--config-file`` option.
3578
If the environment variable is defined and the target file cannot be accessed,
3679
``gitlab`` will fail explicitly.
3780

38-
Content
39-
-------
81+
Configuration file format
82+
-------------------------
4083

4184
The configuration file uses the ``INI`` format. It contains at least a
4285
``[global]`` section, and a specific section for each GitLab server. For

‎gitlab/cli.py‎

Lines changed: 103 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919

2020
import argparse
2121
import functools
22+
import os
2223
import re
2324
import sys
2425
from types import ModuleType
@@ -112,17 +113,25 @@ def _get_base_parser(add_help: bool = True) -> argparse.ArgumentParser:
112113
"-v",
113114
"--verbose",
114115
"--fancy",
115-
help="Verbose mode (legacy format only)",
116+
help="Verbose mode (legacy format only) [env var: GITLAB_VERBOSE]",
116117
action="store_true",
118+
default=os.getenv("GITLAB_VERBOSE"),
117119
)
118120
parser.add_argument(
119-
"-d", "--debug", help="Debug mode (display HTTP requests)", action="store_true"
121+
"-d",
122+
"--debug",
123+
help="Debug mode (display HTTP requests) [env var: GITLAB_DEBUG]",
124+
action="store_true",
125+
default=os.getenv("GITLAB_DEBUG"),
120126
)
121127
parser.add_argument(
122128
"-c",
123129
"--config-file",
124130
action="append",
125-
help="Configuration file to use. Can be used multiple times.",
131+
help=(
132+
"Configuration file to use. Can be used multiple times. "
133+
"[env var: PYTHON_GITLAB_CFG]"
134+
),
126135
)
127136
parser.add_argument(
128137
"-g",
@@ -151,7 +160,86 @@ def _get_base_parser(add_help: bool = True) -> argparse.ArgumentParser:
151160
),
152161
required=False,
153162
)
163+
parser.add_argument(
164+
"--server-url",
165+
help=("GitLab server URL [env var: GITLAB_URL]"),
166+
required=False,
167+
default=os.getenv("GITLAB_URL"),
168+
)
169+
parser.add_argument(
170+
"--ssl-verify",
171+
help=(
172+
"Whether SSL certificates should be validated. [env var: GITLAB_SSL_VERIFY]"
173+
),
174+
required=False,
175+
default=os.getenv("GITLAB_SSL_VERIFY"),
176+
)
177+
parser.add_argument(
178+
"--timeout",
179+
help=(
180+
"Timeout to use for requests to the GitLab server. "
181+
"[env var: GITLAB_TIMEOUT]"
182+
),
183+
required=False,
184+
default=os.getenv("GITLAB_TIMEOUT"),
185+
)
186+
parser.add_argument(
187+
"--api-version",
188+
help=("GitLab API version [env var: GITLAB_API_VERSION]"),
189+
required=False,
190+
default=os.getenv("GITLAB_API_VERSION"),
191+
)
192+
parser.add_argument(
193+
"--per-page",
194+
help=(
195+
"Number of entries to return per page in the response. "
196+
"[env var: GITLAB_PER_PAGE]"
197+
),
198+
required=False,
199+
default=os.getenv("GITLAB_PER_PAGE"),
200+
)
201+
parser.add_argument(
202+
"--pagination",
203+
help=(
204+
"Whether to use keyset or offset pagination [env var: GITLAB_PAGINATION]"
205+
),
206+
required=False,
207+
default=os.getenv("GITLAB_PAGINATION"),
208+
)
209+
parser.add_argument(
210+
"--order-by",
211+
help=("Set order_by globally [env var: GITLAB_ORDER_BY]"),
212+
required=False,
213+
default=os.getenv("GITLAB_ORDER_BY"),
214+
)
215+
parser.add_argument(
216+
"--user-agent",
217+
help=(
218+
"The user agent to send to GitLab with the HTTP request. "
219+
"[env var: GITLAB_USER_AGENT]"
220+
),
221+
required=False,
222+
default=os.getenv("GITLAB_USER_AGENT"),
223+
)
154224

225+
tokens = parser.add_mutually_exclusive_group()
226+
tokens.add_argument(
227+
"--private-token",
228+
help=("GitLab private access token [env var: GITLAB_PRIVATE_TOKEN]"),
229+
required=False,
230+
default=os.getenv("GITLAB_PRIVATE_TOKEN"),
231+
)
232+
tokens.add_argument(
233+
"--oauth-token",
234+
help=("GitLab OAuth token [env var: GITLAB_OAUTH_TOKEN]"),
235+
required=False,
236+
default=os.getenv("GITLAB_OAUTH_TOKEN"),
237+
)
238+
tokens.add_argument(
239+
"--job-token",
240+
help=("GitLab CI job token [env var: CI_JOB_TOKEN]"),
241+
required=False,
242+
)
155243
return parser
156244

157245

@@ -243,13 +331,23 @@ def main() -> None:
243331
"whaction",
244332
"version",
245333
"output",
334+
"fields",
335+
"server_url",
336+
"ssl_verify",
337+
"timeout",
338+
"api_version",
339+
"pagination",
340+
"user_agent",
341+
"private_token",
342+
"oauth_token",
343+
"job_token",
246344
):
247345
args_dict.pop(item)
248346
args_dict = {k: _parse_value(v) for k, v in args_dict.items() if v is not None}
249347

250348
try:
251-
gl = gitlab.Gitlab.from_config(gitlab_id, config_files)
252-
if gl.private_token or gl.oauth_token or gl.job_token:
349+
gl = gitlab.Gitlab.merge_config(vars(options), gitlab_id, config_files)
350+
if gl.private_token or gl.oauth_token:
253351
gl.auth()
254352
except Exception as e:
255353
die(str(e))

‎gitlab/client.py‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
# along with this program. If not, see <http://www.gnu.org/licenses/>.
1717
"""Wrapper for the GitLab API."""
1818

19+
import os
1920
import time
2021
from typing import Any, cast, Dict, List, Optional, Tuple, TYPE_CHECKING, Union
2122

@@ -256,6 +257,87 @@ def from_config(
256257
retry_transient_errors=config.retry_transient_errors,
257258
)
258259

260+
@classmethod
261+
def merge_config(
262+
cls,
263+
options: dict,
264+
gitlab_id: Optional[str] = None,
265+
config_files: Optional[List[str]] = None,
266+
) -> "Gitlab":
267+
"""Create a Gitlab connection by merging configuration with
268+
the following precedence:
269+
270+
1. Explicitly provided CLI arguments,
271+
2. Environment variables,
272+
3. Configuration files:
273+
a. explicitly defined config files:
274+
i. via the `--config-file` CLI argument,
275+
ii. via the `PYTHON_GITLAB_CFG` environment variable,
276+
b. user-specific config file,
277+
c. system-level config file,
278+
4. Environment variables always present in CI (CI_SERVER_URL, CI_JOB_TOKEN).
279+
280+
Args:
281+
options: A dictionary of explicitly provided key-value options.
282+
gitlab_id: ID of the configuration section.
283+
config_files: List of paths to configuration files.
284+
Returns:
285+
(gitlab.Gitlab): A Gitlab connection.
286+
287+
Raises:
288+
gitlab.config.GitlabDataError: If the configuration is not correct.
289+
"""
290+
config = gitlab.config.GitlabConfigParser(
291+
gitlab_id=gitlab_id, config_files=config_files
292+
)
293+
url = (
294+
options.get("server_url")
295+
or config.url
296+
or os.getenv("CI_SERVER_URL")
297+
or gitlab.const.DEFAULT_URL
298+
)
299+
private_token, oauth_token, job_token = cls._merge_auth(options, config)
300+
301+
return cls(
302+
url=url,
303+
private_token=private_token,
304+
oauth_token=oauth_token,
305+
job_token=job_token,
306+
ssl_verify=options.get("ssl_verify") or config.ssl_verify,
307+
timeout=options.get("timeout") or config.timeout,
308+
api_version=options.get("api_version") or config.api_version,
309+
per_page=options.get("per_page") or config.per_page,
310+
pagination=options.get("pagination") or config.pagination,
311+
order_by=options.get("order_by") or config.order_by,
312+
user_agent=options.get("user_agent") or config.user_agent,
313+
)
314+
315+
@staticmethod
316+
def _merge_auth(options: dict, config: gitlab.config.GitlabConfigParser) -> Tuple:
317+
"""
318+
Return a tuple where at most one of 3 token types ever has a value.
319+
Since multiple types of tokens may be present in the environment,
320+
options, or config files, this precedence ensures we don't
321+
inadvertently cause errors when initializing the client.
322+
323+
This is especially relevant when executed in CI where user and
324+
CI-provided values are both available.
325+
"""
326+
private_token = options.get("private_token") or config.private_token
327+
oauth_token = options.get("oauth_token") or config.oauth_token
328+
job_token = (
329+
options.get("job_token") or config.job_token or os.getenv("CI_JOB_TOKEN")
330+
)
331+
332+
if private_token:
333+
return (private_token, None, None)
334+
if oauth_token:
335+
return (None, oauth_token, None)
336+
if job_token:
337+
return (None, None, job_token)
338+
339+
return (None, None, None)
340+
259341
def auth(self) -> None:
260342
"""Performs an authentication using private token.
261343

‎gitlab/config.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
from pathlib import Path
2424
from typing import List, Optional, Union
2525

26-
from gitlab.const import DEFAULT_URL, USER_AGENT
26+
from gitlab.const import USER_AGENT
2727

2828
_DEFAULT_FILES: List[str] = [
2929
"/etc/python-gitlab.cfg",
@@ -119,7 +119,7 @@ def __init__(
119119
self.retry_transient_errors: bool = False
120120
self.ssl_verify: Union[bool, str] = True
121121
self.timeout: int = 60
122-
self.url: str = DEFAULT_URL
122+
self.url: Optional[str] = None
123123
self.user_agent: str = USER_AGENT
124124

125125
self._files = _get_config_files(config_files)

0 commit comments

Comments
 (0)