Skip to content

[Feat] Framework for running live tests#136

Closed
t8y8 wants to merge 4 commits intotableau:developmentfrom
t8y8:feature-livetests
Closed

[Feat] Framework for running live tests#136
t8y8 wants to merge 4 commits intotableau:developmentfrom
t8y8:feature-livetests

Conversation

@t8y8
Copy link
Copy Markdown
Collaborator

@t8y8 t8y8 commented Feb 2, 2017

Here's a basic framework for running tests against a live server.

Initial stab.

Open questions:

  1. What'cha think?
  2. We can have them all in one file (smoke.py) or split them somehow.
    I'm thinking 2 files, smoke.py which is safe to run anywhere because it is stateless. And then another set that will create a site, publish, perform various operations, and then delete it when done

/cc @RussTheAerialist

EDIT: Running from the command line requires you to have the package installed in development mode, I didn't get into any path manipulation because it's icky and I didn't get it to work right/ pip install -e . is simplier

@graysonarts
Copy link
Copy Markdown
Contributor

Overall, it's fine, but we definitely need to build a better infrastructure around storying tests because this is entirely too simple for the amount of tests we will be moving into this system. If you take a look at the current e2e tests for the rest api, you'll see what I mean (I can show you on Monday). Having them all in a single file is not going to be feasible.

@t8y8
Copy link
Copy Markdown
Collaborator Author

t8y8 commented Feb 4, 2017

That's fairly easy to do if we rely on just environment variables -- then it's just another suite that runs under python -m unittest smokeand we can have as many files as we deem appropriate.
It's a pain to make unittest and standalone work with command line arguments, though.

Some options:

  1. We figure out how to make that work. I'm sure it's possible with custom test loaders/runners.
  2. We don't, and happily rely environment variables
  3. We have some configuration file somewhere that we can read variables from for the integration tests (this may be useful if we need to parameterize other tests)
  4. We move to py.test or nosetests which I believe allow more advanced test configuration.

I'm kind of leaning towards 2, which reduces the overhead of this PR to just "Grabbing the env variables and checking in some starter tests"

@t8y8
Copy link
Copy Markdown
Collaborator Author

t8y8 commented May 11, 2017

Subsumed by #176

@t8y8 t8y8 closed this May 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants