Maniphest T192214

Make dumps accessible in PAWS
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Chicocvenancio
	Apr 14 2018, 6:38 PM

Description

Task to define a method and policy to update said method of making dumps accessible in PAWS. At the moment PAWS uses the symbolic link defined at /public/dumps/incr to find what path dumps is and then mounts it to /public/dumps inside each user pod. This still feels a little hacky and prone to problems in the future, but might be the best way handle this until we are free of NFS.

Description was:
At the moment PAWS does mount /public/dumps in the users pods, but since that contains symlinks to the actual mounted NFS shares they are not accessible.

Related Objects

Mentioned In: T167086: Consider moving PAWS to its own Cloud VPS project, rather than using instances inside Toolforge

Event Timeline

Chicocvenancio created this task.Apr 14 2018, 6:38 PM

One option we have is to do a simple volume mount of /mnt/nfs/dumps-labstore1006.wikimedia.org and that should make the symlink work.

• Tbayer subscribed.May 9 2018, 9:44 PM

• bd808 moved this task from Backlog to Dumps on the Data-Services board.May 10 2018, 3:25 PM

Done with host volume to /mnt/nfs/dumps-labstore1006.wikimedia.org. This is non ideal as any changes to that nfs mount structure will break things.

https://github.com/yuvipanda/paws/commit/4a79ac21a7f4807fec0548cdcbecb0f96e650e83

As expected, the change to dumps-labstore1007.wikimedia.org broke this. I went ahead with a, slightly less brittle hack of reading the symlink from '/public/dumps/incr' to find what is the currently used nfs mount for dumps.

Still think this isn't ideal, @chasemp, do we have somewhere we can query for that info beyond the symlinks?

Ivanhercaz subscribed.Feb 2 2019, 5:10 PM

Chicocvenancio moved this task from Backlog to zero-to-jupyterhub-k8s 0.8.0 on the PAWS board.Mar 11 2019, 1:52 PM

Chicocvenancio edited projects, added PAWS (zero-to-jupyterhub-k8s 0.8.0); removed PAWS.

• GTirloni edited projects, added PAWS, cloud-services-team (Kanban); removed PAWS (zero-to-jupyterhub-k8s 0.8.0).Mar 23 2019, 9:28 PM

Chicocvenancio edited projects, added PAWS (zero-to-jupyterhub-k8s 0.8.0); removed PAWS.Mar 23 2019, 9:41 PM

Just to clarify for casual readers: the dumps are currently accessible. (I used them successfully in this notebook earlier this month, thanks @Chicocvenancio for fixing this earlier!) I understand this ticket is now about implementing this in a more future-proof manner - should the task description be updated?

Chicocvenancio mentioned this in T167086: Consider moving PAWS to its own Cloud VPS project, rather than using instances inside Toolforge.Apr 1 2019, 11:32 AM

In T192214#5051459, @Tbayer wrote:

Just to clarify for casual readers: the dumps are currently accessible. (I used them successfully in this notebook earlier this month, thanks @Chicocvenancio for fixing this earlier!) I understand this ticket is now about implementing this in a more future-proof manner - should the task description be updated?

Done, thanks for the reminder.

• bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Jul 11 2019, 4:29 PM

I've set up a pretty solid dumps mount in the new paws cluster at this point, and the symlinks should work as expected (and just like in Toolforge).

Make dumps accessible in PAWSClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Make dumps accessible in PAWS
Closed, ResolvedPublic
Actions