-
-
Notifications
You must be signed in to change notification settings - Fork 9.8k
[Lock] add mongodb store #31889
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Lock] add mongodb store #31889
Conversation
|
Please target |
55526dc to
0e77ec6
Compare
|
rebased from |
|
I've stripped it from
Added in symfony/symfony-docs#11735 |
This PR was merged into the 4.3 branch. Discussion ---------- [Lock] mongodb store removed from symfony 4.3 MongoDbStore wasn't actually added in symfony 4.3, it will be added in 4.4. See symfony/symfony#31889 Commits ------- 75ad033 #27345 Removed Lock MongoDbStore, it will be added in symfony 4.4
|
|
||
| /** | ||
| * MongoDbStore is a StoreInterface implementation using MongoDB as a storage | ||
| * engine. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about adding that it requires an external lib?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pattern public static function isSupported(): bool has been added to MongoDbStore as well as @requires extension mongodb on the class. I've also moved the CAUTION: phpdoc up to the class to be inline with other stores.
753d398 to
f3f8568
Compare
|
I'm looking into the failing test later today |
Simperfit
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice feature !
Status: Needs Work
| * | ||
| * @internal | ||
| */ | ||
| public static function isSupported(): bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imho this is not needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did this because some of Jérémy's stores do it. I thought it might be useful later if a bundle extension needs to check support to auto default/configure something supported, but maybe we could add an interface for this later to all stores if we ever did something like that. Also thinking about it more... you could never auto configure a MongoDbStore since the database name option is mandatory. So I have removed static::isSupported().
| */ | ||
| public function __construct(Client $mongo, array $options, float $initialTtl = 300.0) | ||
| { | ||
| if (!static::isSupported()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can directly check if it is supported here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| * | ||
| * @see https://docs.mongodb.com/manual/applications/replication/ | ||
| */ | ||
| public function __construct(Client $mongo, array $options, float $initialTtl = 300.0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should not type-hint the mongodb client, what happens if he does not exist ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then the developer wouldn't be able to construct a MongoDB\Client. If they pass something that isn't a MongoDB\Client they would get a TypeError:
Uncaught TypeError: Argument 1 passed to Symfony\Component\Lock\Store\MongoDbStore::__construct() must be an instance of MongoDB\Client, x given
This is what I would want to happen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if you don't have MongoDB ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if you don't have mongodb/mongodb you can't construct a MongoDB\Client
if you don't have ext-mongodb you can't composer require mongodb/mongodb
if you somehow have installed mongodb/mongodb and don't have ext-mongodb you will fail to construct a MongoDB\Client (this is mongodb/mongodb's responsibility):
<?php
require __DIR__.'/vendor/autoload.php';
$mongoClient = new MongoDB\Client('mongodb://127.0.0.1');
$store = new Symfony\Component\Lock\Store\MongoDbStore($mongoClient, [
'database' => 'test',
]);$ docker run --rm -it -v $PWD:$PWD -w $PWD php php ./test.php
Fatal error: Uncaught Error: Class 'MongoDB\Driver\Manager' not found in /home/kralos/src/symfony/vendor/mongodb/mongodb/src/Client.php:87
Stack trace:
#0 /home/kralos/src/symfony/test.php(5): MongoDB\Client->__construct('mongodb://127.0...')
#1 {main}
thrown in /home/kralos/src/symfony/vendor/mongodb/mongodb/src/Client.php on line 87
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MongoDB\Driver\Manager is provided by ext-mongodb
MongoDB\Client is provided by mongodb/mongodb
Symfony\Component\Lock\Store\MongoDbStore depends on mongodb/mongodb
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you not type-hint MongoDB\Client for that reason. maybe i'm wrong cc @jderusse
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the if (!\extension_loaded('mongodb')) { check in the constructor since it's impossible to hit it. You can't pass a MongoDB\Client without mongodb/mongodb which you can't have without ext-mongodb.
We still have in the class docblock @requires extension mongodb
|
|
||
| if (!isset($this->options['database'])) { | ||
| throw new InvalidArgumentException( | ||
| 'You must provide the "database" option for MongoDBStore' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should inline this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
|
||
| if ($this->options['gcProbablity'] < 0.0 || $this->options['gcProbablity'] > 1.0) { | ||
| throw new InvalidArgumentException(sprintf( | ||
| '"%s" gcProbablity must be a float from 0.0 to 1.0, "%f" given.', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should inline this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
6df4973 to
a1fe6d0
Compare
|
|
|
This one looks ready for Symfony 4.4/5.0 👍 |
| || (random_int(0, PHP_INT_MAX) / PHP_INT_MAX) <= $this->options['gcProbablity'] | ||
| ) | ||
| ) { | ||
| if ($this->getDatabaseVersion() < '2.2') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works by accident, version_compare should be used for these
>>> '2.10' < '2.2'
=> true
instances: 2
But why are we adding support for legacy version anyway? 2.2 was EOL-ed in 2014. We are paying the price not just with added complexity but with extra query on each request as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'd prefer not to support <2.2. removing support and documenting requirements
| { | ||
| $this->getCollection()->deleteMany([ // filter | ||
| 'expires_at' => [ | ||
| '$lte' => $this->createDateTime(microtime(true)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is inconsistent. IMHO parameter should either be non-optional, or not specified here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed, I've opted for not optional. the objective of the function is to create a MongoDb compliant UTCDateTime to millisecond precision.
| if (1 === \count($writeErrors)) { | ||
| $code = $writeErrors[0]->getCode(); | ||
| } else { | ||
| $code = $e->getCode(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor thing, but else branch could be easily avoided here by specifying this as default. This is super cheap call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated
|
@nicolas-grekas can you let me know if you're happy for this to be merged? I'll rebase/squash etc if everyone is happy. The appveyor CI failure is not Lock component by the way... |
jderusse
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you update the StoreFactory to add the MongoDB\Client and mongodb:// in the list of managed DSN.
Need to update the CHANGELOG.md
d2ac16e to
30d9d2c
Compare
In order to add I have therefore added support for passing a I have also added DSN support to Neither
CI is currently passing (the errors from travis are related to components other then I've left the commit history for review purposes, if we are happy with the current iteration let me know and I can squash (if required). |
|
|
||
| case 0 === strpos($connection, 'flock://'): | ||
| return new FlockStore(substr($connection, 8)); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, something funny happened with my rebase
|
|
||
| if ($this->options['gcProbablity'] < 0.0 || $this->options['gcProbablity'] > 1.0) { | ||
| throw new InvalidArgumentException(sprintf('"%s" gcProbablity must be a float from 0.0 to 1.0, "%f" given.', __METHOD__, $this->options['gcProbablity'])); | ||
| throw new InvalidArgumentException(sprintf('%s() gcProbablity must be a float from 0.0 to 1.0, "%f" given.', __METHOD__, $this->options['gcProbablity'])); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove " in "%f" given too (for consistency)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated
| } | ||
|
|
||
| if (null === $this->collection && !isset($this->options['database'])) { | ||
| throw new InvalidArgumentException(sprintf('%s() requires the "database" option when constructing with a %s', __METHOD__, \is_object($mongo) ? \get_class($mongo) : \gettype($mongo))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
database and collection names could be defined in the DSN (similar to PDO). dsn = mongodb://[username:password@]host1[:port1][,host2[:port2:],...]/[database/][collection] ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The database you see there is the authentication database, not the database to use for data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't it the purpose of the querystring parameter authSource? https://symfony.com/doc/master/bundles/DoctrineMongoDBBundle/installation.html#authentication
anyway, if the path component is a standard to declare the auth Database, we can use custom queryString parameters like mongodb://localhost/?lockDatabase=foo&lockCollection=bar
If we can not create the Store from a DSN, developpers using the FrameworkBundle won't be able to use it from the configuration framework.lock = '%env(LOCK_DNS)% (see symfony/symfony-docs#12523)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't it the purpose of the querystring parameter
authSource?
Actually I just tested this (mongodb 4.2.0) and if both /path and ?authSource are specified /path means application database and ?authSource means authentication database. We are still stuck for getting the collection name.
we can use custom queryString parameters
Seems to be the only way, do you have any objection to us taking ?collection=bar? I feel like if we ever had a conflict it would be because MongoDB is using this param for the exact same purpose.
I'm going to enforce when $mongo is a:
MongoDB\Collection:$options['database']is ignored$options['collection']is ignored
MongoDB\Client:$options['database']is mandatory$options['collection']is mandatory
- MongoDB Connection URI string:
/pathis used otherwise$options['database'], at least one is mandatory?collection=is used otherwise$options['collection'], at least one is mandatory
| $database = parse_url($mongo, PHP_URL_PATH); | ||
| if (null !== $database) { | ||
| $this->options['database'] = $database; | ||
| } | ||
| $query = parse_url($mongo, PHP_URL_QUERY); | ||
| $queryParams = []; | ||
| parse_str($query ?? '', $queryParams); | ||
| $collection = $queryParams['collection'] ?? null; | ||
| if (null !== $collection) { | ||
| $this->options['collection'] = $collection; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's about something similare to Messenger PDO
if (false === $parsedUrl = parse_url($dsn)) {
throw new InvalidArgumentException(sprintf('The given Mongodb DSN "%s" is invalid.', $dsn));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$this->option = $option + $query;
$this->options['database'] = $this->options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You prefer $options to take precedence over /path?collection=?
<?php
declare(strict_types=1);
$mongo = 'mongodb://127.0.0.1/uri-db?collection=uri-col';
$options = [
'database' => 'option-db',
'collection' => 'option-col',
];
if (false === $parsedUrl = parse_url($mongo)) {
throw new \InvalidArgumentException(sprintf('The given MongoDB Connection URI "%s" is invalid.', $mongo));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$options += $query;
$options['database'] = $options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;
print_r($options);
/* Array
(
[database] => option-db
[collection] => option-col
) */There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've implemented this but without polluting $this->options with querystring params other than collection.
if (false === $parsedUrl = parse_url($mongo)) {
throw new InvalidArgumentException(sprintf('The given MongoDB Connection URI "%s" is invalid.', $mongo));
}
$query = [];
if (isset($parsedUrl['query'])) {
parse_str($parsedUrl['query'], $query);
}
$this->options['collection'] = $this->options['collection'] ?? $query['collection'] ?? null;
$this->options['database'] = $this->options['database'] ?? ltrim($parsedUrl['path'] ?? '', '/') ?: null;|
@jderusse happy with current implementation? want me to squash? |
jderusse
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks @kralos
|
@nicolas-grekas can we get this merged? do you need a squash first? I don't want to miss |
af7825d to
a6bfa59
Compare
|
Thank you @kralos. |
ext-mongodbandmongodb/mongodbto test)4.45.1 Doc PRLooks like I messed up
kralos:27345-lock-mongodbwith a force push (trying to fix ci issues) right before it was merged tomaster(4.3.0).see #27648
Description
We should support Semaphore Locks with a MongoDB back end to allow those that already use MongoDB as a distributed storage engine.
Symfony already partially supports MongoDB for session storage:
Symfony\Component\HttpFoundation\Session\Storage\Handler\MongoDbSessionHandlerExample