Skip to content

Commit be35ff7

Browse files
behackettbehackett
committed
PYTHON-719 Read preference backward compatibility
This commit does a few things: - Adds tag_sets back (deprecated) - Adds secondary_acceptable_latency_ms back (deprecated) - Makes acceptable latency a per read preference setting - Cleans up read preference instance generation - Adds latencyThresholdMS as an alias for secondaryAcceptableLatencyMS. The name may change before 3.0 is released.
1 parent 665440b commit be35ff7

File tree

15 files changed

+207
-126
lines changed

15 files changed

+207
-126
lines changed

‎doc/api/pymongo/cursor.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
.. automodule:: pymongo.cursor
55
:synopsis: Tools for iterating over MongoDB query results
66

7-
.. autoclass:: pymongo.cursor.Cursor(collection, spec=None, fields=None, skip=0, limit=0, timeout=True, snapshot=False, tailable=False, sort=None, max_scan=None, as_class=None, await_data=False, partial=False, manipulate=True, read_preference=None, exhaust=False, compile_re=True)
7+
.. autoclass:: pymongo.cursor.Cursor(collection, spec=None, fields=None, skip=0, limit=0, timeout=True, snapshot=False, tailable=False, sort=None, max_scan=None, as_class=None, await_data=False, partial=False, manipulate=True, read_preference=None, tag_sets=None, secondary_acceptable_latency_ms=None, exhaust=False, compile_re=True)
88
:members:
99

1010
.. describe:: c[index]

‎doc/examples/high_availability.rst‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,7 @@ per-query basis, e.g.::
200200
>>> db.collection.find_one(read_preference=ReadPreference.PRIMARY)
201201

202202
Reads are configured using three options: **read_preference**, **tag_sets**,
203-
and **acceptableLatencyMS**.
203+
and *latency_threshold_ms**.
204204

205205
**read_preference**:
206206

@@ -210,18 +210,18 @@ and **acceptableLatencyMS**.
210210

211211
- ``PRIMARY_PREFERRED``: Read from the primary if available, or if there is
212212
none, read from a secondary matching your choice of ``tag_sets`` and
213-
``acceptableLatencyMS``.
213+
``latency_threshold_ms``.
214214

215215
- ``SECONDARY``: Read from a secondary matching your choice of ``tag_sets`` and
216-
``acceptableLatencyMS``. If no matching secondary is available,
216+
``latency_threshold_ms``. If no matching secondary is available,
217217
raise :class:`~pymongo.errors.AutoReconnect`.
218218

219219
- ``SECONDARY_PREFERRED``: Read from a secondary matching your choice of
220-
``tag_sets`` and ``acceptableLatencyMS`` if available, otherwise
220+
``tag_sets`` and ``latency_threshold_ms`` if available, otherwise
221221
from primary (regardless of the primary's tags and latency).
222222

223223
- ``NEAREST``: Read from any member matching your choice of ``tag_sets`` and
224-
``acceptableLatencyMS``.
224+
``latency_threshold_ms``.
225225

226226
**tag_sets**:
227227

@@ -251,18 +251,18 @@ from any member that matches the mode, ignoring tags."
251251

252252
See :mod:`~pymongo.read_preferences` for more information.
253253

254-
**acceptableLatencyMS**:
254+
**latency_threshold_ms**:
255255

256256
If multiple members match the mode and tag sets, MongoReplicaSetClient reads
257257
from among the nearest members, chosen according to ping time. By default,
258258
only members whose ping times are within 15 milliseconds of the nearest
259259
are used for queries. You can choose to distribute reads among members with
260-
higher latencies by setting ``acceptableLatencyMS`` to a larger
260+
higher latencies by setting ``latency_threshold_ms`` to a larger
261261
number. In that case, MongoReplicaSetClient distributes reads among matching
262-
members within ``acceptableLatencyMS`` of the closest member's
262+
members within ``latency_threshold_ms`` of the closest member's
263263
ping time.
264264

265-
.. note:: ``acceptableLatencyMS`` is ignored when talking to a
265+
.. note:: ``latency_threshold_ms`` is ignored when talking to a
266266
replica set *through* a mongos. The equivalent is the localThreshold_ command
267267
line option.
268268

‎gridfs/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,8 @@ def find(self, *args, **kwargs):
324324
examined when performing the query
325325
- `read_preference` (optional): The read preference for
326326
this query.
327+
- `tag_sets` **DEPRECATED**
328+
- `secondary_acceptable_latency_ms` **DEPRECATED**
327329
- `compile_re` (optional): if ``False``, don't attempt to compile
328330
BSON regex objects into Python regexes. Return instances of
329331
:class:`~bson.regex.Regex` instead.

‎gridfs/grid_file.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -628,7 +628,8 @@ class GridOutCursor(Cursor):
628628
"""
629629
def __init__(self, collection, spec=None, skip=0, limit=0,
630630
timeout=True, sort=None, max_scan=None,
631-
read_preference=None, compile_re=True):
631+
read_preference=None, tag_sets=None,
632+
secondary_acceptable_latency_ms=None, compile_re=True):
632633
"""Create a new cursor, similar to the normal
633634
:class:`~pymongo.cursor.Cursor`.
634635
@@ -648,7 +649,8 @@ def __init__(self, collection, spec=None, skip=0, limit=0,
648649
super(GridOutCursor, self).__init__(
649650
collection.files, spec, skip=skip, limit=limit, timeout=timeout,
650651
sort=sort, max_scan=max_scan, read_preference=read_preference,
651-
compile_re=compile_re)
652+
secondary_acceptable_latency_ms=secondary_acceptable_latency_ms,
653+
tag_sets=tag_sets, compile_re=compile_re)
652654

653655
def next(self):
654656
"""Get next GridOut object from cursor.

‎pymongo/collection.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -823,6 +823,8 @@ def find(self, *args, **kwargs):
823823
outgoing SON manipulators before returning.
824824
- `read_preference` (optional): The read preference for
825825
this query.
826+
- `tag_sets` **DEPRECATED**
827+
- `secondary_acceptable_latency_ms` **DEPRECATED**
826828
- `compile_re` (optional): if ``False``, don't attempt to compile
827829
BSON regex objects into Python regexes. Return instances of
828830
:class:`~bson.regex.Regex` instead.
@@ -857,7 +859,8 @@ def find(self, *args, **kwargs):
857859
version **>= 1.5.1**
858860
859861
.. versionchanged:: 3.0
860-
Removed the `network_timeout`, `tag_sets`, and
862+
Removed the `network_timeout` parameter.
863+
Deprecated the `tag_sets`, and
861864
`secondary_acceptable_latency_ms` parameters.
862865
863866
.. versionadded:: 2.7

‎pymongo/common.py‎

Lines changed: 42 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,16 @@
1414

1515

1616
"""Functions and classes common to multiple pymongo modules."""
17+
1718
import sys
18-
from pymongo import read_preferences
19+
import warnings
1920

2021
from pymongo.auth import MECHANISMS
2122
from pymongo.errors import ConfigurationError
23+
from pymongo.read_preferences import (make_read_preference,
24+
read_pref_mode_from_name,
25+
ReadPreference,
26+
ServerMode)
2227
from pymongo.write_concern import WriteConcern
2328
from bson.binary import (OLD_UUID_SUBTYPE, UUID_SUBTYPE,
2429
JAVA_LEGACY, CSHARP_LEGACY)
@@ -194,7 +199,7 @@ def validate_timeout_or_none(option, value):
194199
def validate_read_preference(dummy, value):
195200
"""Validate a read preference.
196201
"""
197-
if not isinstance(value, read_preferences.ServerMode):
202+
if not isinstance(value, ServerMode):
198203
raise ConfigurationError("%r is not a "
199204
"valid read preference." % (value,))
200205
return value
@@ -204,7 +209,7 @@ def validate_read_preference_mode(dummy, name):
204209
"""Validate read preference mode for a MongoReplicaSetClient.
205210
"""
206211
try:
207-
return read_preferences.read_pref_mode_from_name(name)
212+
return read_pref_mode_from_name(name)
208213
except ValueError:
209214
raise ConfigurationError("Not a valid read preference")
210215

@@ -278,7 +283,8 @@ def validate_read_preference_tags(name, value):
278283
'read_preference': validate_read_preference,
279284
'readpreference': validate_read_preference_mode,
280285
'readpreferencetags': validate_read_preference_tags,
281-
'acceptablelatencyms': validate_positive_float,
286+
'latencythresholdms': validate_positive_float,
287+
'secondaryacceptablelatencyms': validate_positive_float,
282288
'auto_start_request': validate_boolean,
283289
'use_greenlets': validate_boolean,
284290
'authmechanism': validate_auth_mechanism,
@@ -329,25 +335,24 @@ class BaseObject(object):
329335

330336
def __init__(self, **options):
331337

332-
self.__read_pref = read_preferences.ReadPreference.PRIMARY
338+
self.__read_pref = None
333339
self.__uuid_subtype = OLD_UUID_SUBTYPE
334340
self.__write_concern = None
335341
self.__set_options(options)
336342

343+
if 'read_preference' not in options:
344+
mode = options.get('readpreference', 0)
345+
latency = options.get('secondaryacceptablelatencyms',
346+
options.get('latencythresholdms', 15))
347+
tags = options.get('readpreferencetags')
348+
self.__read_pref = make_read_preference(mode, latency, tags)
349+
337350
def __set_options(self, options):
338351
"""Validates and sets all options passed to this object."""
339352
wc_opts = {}
340353
for option, value in iteritems(options):
341354
if option == 'read_preference':
342355
self.__read_pref = validate_read_preference(option, value)
343-
elif option == 'readpreference':
344-
klass = read_preferences.read_pref_class_from_mode(value)
345-
if value == 0:
346-
# Primary, no tags
347-
self.__read_pref = klass()
348-
continue
349-
tags = options.get('readpreferencetags', None)
350-
self.__read_pref = klass(tags)
351356
elif option == 'uuidrepresentation':
352357
self.__uuid_subtype = validate_uuid_subtype(option, value)
353358
elif option in WRITE_CONCERN_OPTIONS:
@@ -436,6 +441,30 @@ def __set_read_pref(self, value):
436441

437442
read_preference = property(__get_read_pref, __set_read_pref)
438443

444+
def __get_latency(self):
445+
return self.__read_pref.latency_threshold_ms
446+
447+
def __set_latency(self, latency):
448+
warnings.warn("The secondary_acceptable_latency_ms attribute is "
449+
"deprecated", DeprecationWarning, stacklevel=2)
450+
mode = self.__read_pref.mode
451+
tag_sets = self.__read_pref.tag_sets
452+
self.__read_pref = make_read_preference(mode, latency, tag_sets)
453+
454+
secondary_acceptable_latency_ms = property(__get_latency, __set_latency)
455+
456+
def __get_tags(self):
457+
return self.__read_pref.tag_sets
458+
459+
def __set_tags(self, tag_sets):
460+
warnings.warn("The tag_sets attribute is deprecated",
461+
DeprecationWarning, stacklevel=2)
462+
mode = self.__read_pref.mode
463+
latency = self.__read_pref.latency_threshold_ms
464+
self.__read_pref = make_read_preference(mode, latency, tag_sets)
465+
466+
tag_sets = property(__get_tags, __set_tags)
467+
439468
def __get_uuid_subtype(self):
440469
"""This attribute specifies which BSON Binary subtype is used when
441470
storing UUIDs. Historically UUIDs have been stored as BSON Binary

‎pymongo/cursor.py‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@
1414

1515
"""Cursor class to iterate over Mongo query results."""
1616
import copy
17+
import warnings
18+
1719
from collections import deque
1820

1921
from bson import RE_TYPE
@@ -71,8 +73,9 @@ def __init__(self, collection, spec=None, fields=None, skip=0, limit=0,
7173
timeout=True, snapshot=False, tailable=False, sort=None,
7274
max_scan=None, as_class=None,
7375
await_data=False, partial=False, manipulate=True,
74-
read_preference=None, exhaust=False, compile_re=True,
75-
_uuid_subtype=None):
76+
read_preference=None, tag_sets=None,
77+
secondary_acceptable_latency_ms=None,
78+
exhaust=False, compile_re=True, _uuid_subtype=None):
7679
"""Create a new cursor.
7780
7881
Should not be called directly by application developers - see
@@ -148,7 +151,6 @@ def __init__(self, collection, spec=None, fields=None, skip=0, limit=0,
148151
self.__comment = None
149152
self.__as_class = as_class
150153
self.__manipulate = manipulate
151-
self.__read_preference = read_preference or collection.read_preference
152154
self.__tz_aware = collection.database.connection.tz_aware
153155
self.__compile_re = compile_re
154156
self.__uuid_subtype = _uuid_subtype or collection.uuid_subtype
@@ -158,10 +160,21 @@ def __init__(self, collection, spec=None, fields=None, skip=0, limit=0,
158160
self.__retrieved = 0
159161
self.__killed = False
160162

163+
self.__read_preference = read_preference or collection.read_preference
164+
if secondary_acceptable_latency_ms or tag_sets:
165+
warnings.warn("The secondary_acceptable_latency_ms "
166+
"and tag_sets options are deprecated",
167+
DeprecationWarning, stacklevel=3)
168+
mode = self.__read_preference.mode
169+
tags = tag_sets or self.__read_preference.tag_sets
170+
latency = (secondary_acceptable_latency_ms or
171+
self.__read_preference.latency_threshold_ms)
172+
self.__read_preference = make_read_preference(mode, latency, tags)
173+
161174
self.__query_flags = 0
162175
if tailable:
163176
self.__query_flags |= _QUERY_OPTIONS["tailable_cursor"]
164-
if self.__read_preference != ReadPreference.PRIMARY:
177+
if self.__read_preference.mode != ReadPreference.PRIMARY.mode:
165178
self.__query_flags |= _QUERY_OPTIONS["slave_okay"]
166179
if not timeout:
167180
self.__query_flags |= _QUERY_OPTIONS["no_timeout"]
@@ -300,7 +313,7 @@ def __query_spec(self):
300313
# PRIMARY to avoid problems with mongos versions that
301314
# don't support read preferences.
302315
if (self.__collection.database.connection.is_mongos and
303-
self.__read_preference != ReadPreference.PRIMARY):
316+
self.__read_preference.mode != ReadPreference.PRIMARY.mode):
304317

305318
# For maximum backwards compatibility, don't set $readPreference
306319
# for SECONDARY_PREFERRED unless tags are in use. Just rely on

‎pymongo/database.py‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -295,25 +295,36 @@ def _command(self, command, value=1,
295295
fields = helpers._fields_list_to_dict(fields)
296296
command.update(kwargs)
297297

298-
orig = mode = read_preference or self.read_preference
298+
orig = pref = read_preference or self.read_preference
299+
latency = kwargs.pop('secondary_acceptable_latency_ms', None)
300+
tags = kwargs.pop('tags_sets', None)
301+
if latency or tags:
302+
warnings.warn("The secondary_acceptable_latency_ms "
303+
"and tag_sets options are deprecated",
304+
DeprecationWarning, stacklevel=3)
305+
mode = orig.mode
306+
tags = tags or orig.tag_sets
307+
latency = latency or orig.latency_threshold_ms
308+
orig = make_read_preference(mode, latency, tags)
309+
299310
if command_name not in SECONDARY_OK_COMMANDS:
300-
mode = ReadPreference.PRIMARY
311+
pref = ReadPreference.PRIMARY
301312

302313
# Special-case: mapreduce can go to secondaries only if inline
303314
elif command_name == 'mapreduce':
304315
out = command.get('out')
305316
if not isinstance(out, dict) or not out.get('inline'):
306-
mode = ReadPreference.PRIMARY
317+
pref = ReadPreference.PRIMARY
307318

308319
# Special-case: aggregate with $out cannot go to secondaries.
309320
elif command_name == 'aggregate':
310321
for stage in command.get('pipeline', []):
311322
if '$out' in stage:
312-
mode = ReadPreference.PRIMARY
323+
pref = ReadPreference.PRIMARY
313324
break
314325

315326
# Warn if mode will override read_preference.
316-
if mode != orig:
327+
if pref.mode != orig.mode:
317328
warnings.warn("%s does not support %s read preference "
318329
"and will be routed to the primary instead." %
319330
(command_name, orig.name), UserWarning, stacklevel=3)
@@ -323,7 +334,7 @@ def _command(self, command, value=1,
323334
fields=fields,
324335
limit=-1,
325336
as_class=as_class,
326-
read_preference=mode,
337+
read_preference=pref,
327338
compile_re=compile_re,
328339
_uuid_subtype=uuid_subtype)
329340
for doc in cursor:
@@ -391,11 +402,13 @@ def command(self, command, value=1,
391402
Python-incompatible regular expressions, for example from
392403
``currentOp``
393404
- `read_preference`: The read preference for this operation.
405+
- `tag_sets` **DEPRECATED**
406+
- `secondary_acceptable_latency_ms` **DEPRECATED**
394407
- `**kwargs` (optional): additional keyword arguments will
395408
be added to the command document before it is sent
396409
397410
.. versionchanged:: 3.0
398-
Removed the `tag_sets` and `secondary_acceptable_latency_ms`
411+
Deprecated the `tag_sets` and `secondary_acceptable_latency_ms`
399412
options.
400413
.. versionchanged:: 2.7
401414
Added ``compile_re`` option.

0 commit comments

Comments
 (0)