A Simple Form
At the end of the last chapter, we were left with the thought that there was too much duplication in the validation handling bits of our views. Django encourages you to use form classes to do the work of validating user input, and choosing what error messages to display.
We’ll use tests to explore the way Django forms work, and then we’ll refactor our views to use them. As we go along, we’ll see our unit tests and functional tests, in combination, will protect us from regressions.
Moving Validation Logic Into a Form
| In Django, a complex view is a code smell. Could some of that logic be pushed out to a form? Or to some custom methods on the model class? Or (perhaps best of all) to a non-Django module that represents your business logic? |
Forms have several superpowers in Django:
-
They can process user input and validate it for errors.
-
They can be used in templates to render HTML input elements, and error messages too.
-
And, as we’ll see later, some of them can even save data to the database for you.
You don’t have to use all three superpowers in every form. You may prefer to roll your own HTML or do your own saving. But they are an excellent place to keep validation logic.
Exploring the Forms API with a Unit Test
Let’s do a little experimenting with forms by using a unit test. My plan is to iterate towards a complete solution, and hopefully introduce forms gradually enough that they’ll make sense if you’ve never seen them before.
First we add a new file for our form unit tests, and we start with a test that just looks at the form HTML:
from django.test import TestCase
from lists.forms import ItemForm
class ItemFormTest(TestCase):
def test_form_renders_item_text_input(self):
form = ItemForm()
self.fail(form.as_p())form.as_p() renders the form as HTML. This unit test uses a self.fail
for some exploratory coding. You could just as easily use a manage.py shell
session, although you’d need to keep reloading your code for each change.
Let’s make a minimal form. It inherits from the base Form class, and has
a single field called item_text:
from django import forms
class ItemForm(forms.Form):
item_text = forms.CharField()We now see a failure message that tells us what the autogenerated form HTML will look like:
AssertionError: <p>
<label for="id_item_text">Item text:</label>
<input type="text" name="item_text" required id="id_item_text">
[...]
</p>
It’s already pretty close to what we have in base.html. We’re missing the placeholder attribute and the Bootstrap CSS classes. Let’s make our unit test into a test for that:
class ItemFormTest(TestCase):
def test_form_item_input_has_placeholder_and_css_classes(self):
form = ItemForm()
rendered = form.as_p()
self.assertIn('placeholder="Enter a to-do item"', rendered)
self.assertIn('class="form-control form-control-lg"', rendered)That gives us a fail, which justifies some real coding:
self.assertIn('placeholder="Enter a to-do item"', rendered)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'placeholder="Enter a to-do item"' not found in [...]
How can we customise the input for a form field? We use a "widget". Here it is with just the placeholder:
class ItemForm(forms.Form):
item_text = forms.CharField(
widget=forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
}
),
)That gives:
AssertionError: 'class="form-control form-control-lg"' not found in '<p>\n <label for="id_item_text">Item text:</label>\n <input type="text" name="item_text" placeholder="Enter a to-do item" required id="id_item_text">\n \n \n \n \n </p>'
And then:
widget=forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),
Doing this sort of widget customisation would get tedious
if we had a much larger, more complex form.
Check out
django-crispy-forms
for some help.
|
Switching to a Django ModelForm
What’s next?
We want our form to reuse the validation code that we’ve already defined on our model.
Django provides a special class that can autogenerate a form for a model, called ModelForm.
As you’ll see, it’s configured using a special inner class called Meta:
from django import forms
from lists.models import Item
class ItemForm(forms.models.ModelForm):
class Meta: (1)
model = Item
fields = ("text",)
# item_text = forms.CharField( (2)
# widget=forms.widgets.TextInput(
# attrs={
# "placeholder": "Enter a to-do item",
# "class": "form-control form-control-lg",
# }
# ),
# )| 1 | In Meta, we specify which model the form is for
and which fields we want it to use. |
| 2 | We’ll comment out our manually created field for now. |
ModelForm does all sorts of smart stuff,
like assigning sensible HTML form input types to different types of field,
and applying default validation.
Check out the
docs
for more info.
We now have some different-looking form HTML:
AssertionError: 'placeholder="Enter a to-do item"' not found in '<p>\n <label for="id_text">Text:</label>\n <textarea name="text" cols="40" rows="10" required id="id_text">\n</textarea>\n \n \n \n \n </p>'
It’s lost our placeholder and CSS class.
And you can also see that it’s using
name="text" instead of name="item_text".
We can probably live with that.
But it’s using a textarea instead of a normal input,
and that’s not the UI we want for our app.
Thankfully, you can override widgets for ModelForm fields,
similarly to the way we did it with the normal form:
class ItemForm(forms.models.ModelForm):
class Meta:
model = Item
fields = ("text",)
widgets = { (1)
"text": forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),
}| 1 | We restore some of our commented-out code here, but modified slightly, from being an attribute declaration to a key in a dict. |
That gets the test passing.
Testing and Customising Form Validation
Now let’s see if the ModelForm has picked up the same validation rules
that we defined on the model.
We’ll also learn how to pass data into the form, as if it came from the user:
def test_form_item_input_has_placeholder_and_css_classes(self):
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
form.save()That gives us:
ValueError: The Item could not be created because the data didn't validate.
Good: the form won’t allow you to save if you give it an empty item text. Now let’s see if we can get it to use the specific error message that we
want. The API for checking form validation before we try to save any
data is a function called is_valid:
def test_form_item_input_has_placeholder_and_css_classes(self):
[...]
def test_form_validation_for_blank_items(self):
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
self.assertFalse(form.is_valid())
self.assertEqual(form.errors["text"], ["You can't have an empty list item"])Calling form.is_valid() returns True or False,
but it also has the side effect of validating the input data
and populating the errors attribute.
It’s a dictionary mapping the names of fields to lists of errors for those fields
(it’s possible for a field to have more than one error).
That gives us:
AssertionError: ['This field is required.'] != ["You can't have an empty list item"]
Django already has a default error message
that we could present to the user—you might use it
if you were in a hurry to build your web app,
but we care enough to make our message special.
Customising it means changing error_messages—another Meta variable:
class Meta:
model = Item
fields = ("text",)
widgets = {
"text": forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),
}
error_messages = {"text": {"required": "You can't have an empty list item"}}OK
You know what would be even better than messing about with all these error strings? Having a constant:
EMPTY_ITEM_ERROR = "You can't have an empty list item"
[...]
error_messages = {"text": {"required": EMPTY_ITEM_ERROR}}Rerun the tests to see that they pass…OK. Now we can change the tests too.
from lists.forms import EMPTY_ITEM_ERROR, ItemForm
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
self.assertFalse(form.is_valid())
self.assertEqual(form.errors["text"], [EMPTY_ITEM_ERROR])| This is a good example of reusing constants in tests. It makes it easier to change the error message later. |
And the tests still pass:
OK
Great. Totes committable:
$ git status # should show forms.py and test_forms.py $ git add src/lists $ git commit -m "new form for list items"
Attempting to Use the Form in Our Views
At this point, we may be tempted to carry on—perhaps extend the form to capture uniqueness validation and empty-item validation.
But there’s a sort of corollary to the "deploy as early as possible" Lean methodology, which is "merge code as early as possible". In other words: while building this bit of forms code, it would be easy to go on for ages, adding more and more functionality to the form—I should know, because that’s exactly what I did during the drafting of this chapter, and I ended up doing all sorts of work making an all-singing, all-dancing form class before I realised it wouldn’t actually work for our most basic use case.
So, instead, try to use your new bit of code as soon as possible. This makes sure you never have unused bits of code lying around, and that you start checking your code against "the real world" as soon as possible.
We have a form class that can render some HTML and do validation of at least one kind of error—let’s start using it! We should be able to use it in our base.html template—so, also, in all of our views.
Using the Form in a View with a GET Request
So, let’s start using our form in our home page view:
[...]
from lists.forms import ItemForm
from lists.models import Item, List
def home_page(request):
return render(request, "home.html", {"form": ItemForm()})OK, now let’s try using it in the template—we
replace the old <input ..> with {{ form.text }}:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }} (1)
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
</form>| 1 | {{ form.text }} renders just the HTML input for the text field of the form. |
That causes our two unit tests that check on the form input to fail:
[...]
======================================================================
FAIL: test_renders_input_form
(lists.tests.test_views.HomePageTest.test_renders_input_form)
---------------------------------------------------------------------
Traceback (most recent call last):
File "...goat-book/src/lists/tests/test_views.py", line 19, in
test_renders_input_form
self.assertIn("item_text", [input.get("name") for input in inputs])
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'item_text' not found in ['text', 'csrfmiddlewaretoken'] (1)
======================================================================
FAIL: test_renders_input_form
(lists.tests.test_views.ListViewTest.test_renders_input_form)
---------------------------------------------------------------------
Traceback (most recent call last):
File "...goat-book/src/lists/tests/test_views.py", line 60, in
test_renders_input_form
self.assertIn("item_text", [input.get("name") for input in inputs])
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'item_text' not found in ['csrfmiddlewaretoken'] (2)
Ran 18 tests in 0.022s
FAILED (failures=2)
| 1 | The test for the home page is failing because the name attribute
of the input box is now text, not item_text. |
| 2 | The test for the list view is failing because
because we’re not instantiating a form in that view,
so there’s no form variable in the template.
The input box isn’t even being rendered. |
Let’s fix things one at a time.
First, let’s back out our change and restore the hand-crafted HTML input
in cases where {{ form }} is not defined:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{% if form %}
{{ form.text }}
{% else %}
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
name="item_text"
id="id_new_item"
placeholder="Enter a to-do item"
/>
{% endif %}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
</form>That takes us down to one failure:
AssertionError: 'item_text' not found in ['text', 'csrfmiddlewaretoken']
Let’s make a note to come back and tidy this up, and then we’ll talk about what’s happened and how to deal with it:
The Trade-offs of Django ModelForms: The Frontend Is Coupled to the Database
This highlights one of the trade-offs of using ModelForm:
by auto-generating the form from the model,
we tie the name= attribute of our form’s HTML <input>
to the name of the model field in the database.
In a simple CRUD (create, read, update, and delete) app like ours, that’s probably a good deal.
But it does mean we need to go back and change our assumptions about
what the name= attribute of the input box is going to be.
While we’re at it, it’s worth doing an FT run too:
$ python src/manage.py test functional_tests [...] selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="id_new_item"]; [...] [...] FAILED (errors=4)
Looks like something else has changed.
If you pause the FTs or inspect the HTML manually in a browser,
you’ll see that the ModelForm also changes the id attribute
to being id_text.[1]
A Big Find-and-Replace
If we want to change our assumption about these two attributes, we’ll need to embark on a couple of big find-and-replaces basically:
But before we do that, let’s back out the rest of our changes and get back to a working state.
Backing Out Our Changes and Getting to a Working State
The simplest way to back out changes is with git.
But in this case, leaving a couple of placeholders does no harm,
and they’ll be helpful to come back to later.
So we can leave the {{ form.text }} in the HTML
but, by backing out the change in the view, we’ll make sure that branch is never actually exercised.
Again, to leave ourselves a little placeholder,
we’ll comment out our code rather than deleting it:
def home_page(request):
# return render(request, "home.html", {"form": ItemForm()})
return render(request, "home.html")
Be very cautious about leaving commented-out code
and unused if branches lying around.
Do so only if you’re sure you’re coming back to them very soon,
otherwise your codebase will soon get messy!
|
Now we can do a full unit test and FT run to confirm we’re back to a working state:
$ python src/manage.py test lists Found 18 test(s). [...] OK $ python src/manage.py test functional_tests Found 4 test(s). [...] OK
And let’s do a commit to be able to separate out the rename from anything else:
$ git diff # changes in base.html + views.py $ git commit -am "Placeholders for using form in view+template, not in use yet"
And pop an item on the to-do list:
Renaming the name Attribute
So, let’s have a look for item_text in the codebase:
$ grep -Ir item_text src
src/lists/migrations/0003_list.py: ("lists", "0002_item_text"),
src/lists/tests/test_views.py: self.assertIn("item_text",
[input.get("name") for input in inputs])
[...]
src/lists/templates/base.html: name="item_text"
src/lists/views.py: item = Item(text=request.POST["item_text"], list=nulist)
src/lists/views.py: item = Item(text=request.POST["item_text"],
list=our_list)
We can ignore the migration, which is just using item_text as metadata.
So the changes we need to make are in three places:
-
views.py
-
test_views.py
-
base.html
Let’s go ahead and make those. I’m sure you can manage your own find-and-replace! They should look something like this:
@@ -16,12 +16,12 @@ class HomePageTest(TestCase):
[form] = parsed.cssselect("form[method=POST]")
self.assertEqual(form.get("action"), "/lists/new")
inputs = form.cssselect("input")
- self.assertIn("item_text", [input.get("name") for input in inputs])
+ self.assertIn("text", [input.get("name") for input in inputs])
class NewListTest(TestCase):
def test_can_save_a_POST_request(self):
- self.client.post("/lists/new", data={"item_text": "A new list item"})
+ self.client.post("/lists/new", data={"text": "A new list item"})
self.assertEqual(Item.objects.count(), 1)
new_item = Item.objects.get()
self.assertEqual(new_item.text, "A new list item")
[...]Or, in views.py:
@@ -12,7 +12,7 @@ def home_page(request):
def new_list(request):
nulist = List.objects.create()
- item = Item(text=request.POST["item_text"], list=nulist)
+ item = Item(text=request.POST["text"], list=nulist)
try:
item.full_clean()
item.save()
@@ -29,7 +29,7 @@ def view_list(request, list_id):
if request.method == "POST":
try:
- item = Item(text=request.POST["item_text"], list=our_list)
+ item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)Finally, in base.html:
@@ -21,7 +21,7 @@
{% else %}
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
- name="item_text"
+ name="text"
id="id_new_item"
placeholder="Enter a to-do item"
/>Once you’re done, rerun the unit tests to confirm that the application is self-consistent:
$ python src/manage.py test lists [...] Ran 18 tests in 0.126s OK
And rerun the FTs too:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Good! One down:
Renaming the id Attribute
Now for the id= attribute.
A quick grep shows us that id_new_item appears in the template,
and in all three FT files:
$ grep -r id_new_item
src/lists/templates/base.html: id="id_new_item"
src/functional_tests/test_list_item_validation.py:
self.browser.find_element(By.ID, "id_new_item").send_keys(Keys.ENTER)
src/functional_tests/test_list_item_validation.py:
self.browser.find_element(By.ID, "id_new_item").send_keys("Purchase milk")
[...]
That’s a good call for a refactor within the FTs too. Let’s make a new helper method in base.py:
class FunctionalTest(StaticLiveServerTestCase):
[...]
def get_item_input_box(self):
return self.browser.find_element(By.ID, "id_new_item") (1)| 1 | We’ll keep the old id for now. Working state to working state! |
And then we use it throughout—I had to make four changes in test_simple_list_creation.py, two in test_layout_and_styling.py, and six in test_list_item_validation.py, for example:
# She is invited to enter a to-do item straight away
inputbox = self.get_item_input_box()Or:
# an empty list item. She hits Enter on the empty input box
self.browser.get(self.live_server_url)
self.get_item_input_box().send_keys(Keys.ENTER)I won’t show you every single one; I’m sure you can manage this for yourself!
You can redo the grep to check that you’ve caught them all:
$ grep -r id_new_item src/lists/templates/base.html: id="id_new_item" src/functional_tests/base.py: return self.browser.find_element(By.ID, "id_new_item")
And we can do an FT run too, to make sure we haven’t broken anything:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Good! FT refactor complete—now hopefully we can make
the application-level refactor of the id attribute in just two places,
and we’ve been in a working state the whole way through.
In the FT helper method:
@@ -43,4 +43,4 @@ class FunctionalTest(StaticLiveServerTestCase):
time.sleep(0.5)
def get_item_input_box(self):
- return self.browser.find_element(By.ID, "id_new_item")
+ return self.browser.find_element(By.ID, "id_text")And in the template:
@@ -22,7 +22,7 @@
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
name="text"
- id="id_new_item"
+ id="id_text"
placeholder="Enter a to-do item"
/>
{% endif %}And an FT run to confirm:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Hooray!
A Second Attempt at Using the Form in Our Views
Now that we’ve done the groundwork,
hopefully we can drop in our form in the home_page() once again:
def home_page(request):
return render(request, "home.html", {"form": ItemForm()})Looking good!
$ python src/manage.py test lists Found 18 test(s). [...] OK
Let’s see what happens if we remove that if from the template:
@@ -16,16 +16,7 @@
<h1 class="display-1 mb-4">{% block header_text %}{% endblock %}</h1>
<form method="POST" action="{% block form_action %}{% endblock %}" >
- {% if form %}
- {{ form.text }}
- {% else %}
- <input
- class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
- name="text"
- id="id_text"
- placeholder="Enter a to-do item"
- />
- {% endif %}
+ {{ form.text }}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>Aha—the unit tests are there to tell us
that we need to use the form in view_list() too:
AssertionError: 'text' not found in ['csrfmiddlewaretoken']
Here’s the minimal use of the form—we won’t use it for validation yet, just for getting the form into the template:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
error = None
form = ItemForm()
if request.method == "POST":
try:
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
except ValidationError:
error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error}
)And the tests are happy with that too:
$ python src/manage.py test lists Found 18 test(s). [...] OK
We’re done with the template; what’s next?
Right, let’s move on to the next view that doesn’t use our form yet—new_list().
And actually, that’ll help us with the first item,
which was the whole point of this adventure, really: to
see if the forms can help us better handle validation.
Let’s see how that works now.
Using the Form in a View That Takes POST Requests
Here’s how we can use the form in the new_list() view,
avoiding all the manual manipulation of request.POST and the error message:
def new_list(request):
form = ItemForm(data=request.POST) (1)
if form.is_valid(): (2)
nulist = List.objects.create()
Item.objects.create(text=request.POST["text"], list=nulist)
return redirect(nulist)
else:
return render(request, "home.html", {"form": form}) (3)| 1 | We pass the request.POST data into the form’s constructor. |
| 2 | We use form.is_valid() to determine whether this is a good
or a bad submission. |
| 3 | In the invalid case, we pass the form down to the template, instead of our hardcoded error string. |
That view is now looking much nicer!
But, we have a regression in the unit tests:
======================================================================
FAIL: test_validation_errors_are_sent_back_to_home_page_template (lists.tests.t
est_views.NewListTest.test_validation_errors_are_sent_back_to_home_page_templat
e)
---------------------------------------------------------------------
[...]
self.assertContains(response, expected_error)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true : Couldn't find 'You can't have an empty
list item' in the following response
b'<!doctype html>\n<html lang="en">\n\n <head>\n <title>To-Do
[...]
Using the Form to Display Errors in the Template
We’re failing because we’re not yet using the form to display errors in the template. Here’s how to do that:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
{% if form.errors %} (1)
<div class="invalid-feedback">{{ form.errors.text }}</div> (2)
{% endif %}
</form>| 1 | We change the if to look at form.errors:
it contains a list of all the errors for the form. |
| 2 | form.errors.text is magical Django template syntax
for form.errors["text"]—i.e., the list of errors for the text field in particular. |
What does that do to our unit tests?
====================================================================== FAIL: test_validation_errors_end_up_on_lists_page (lists.tests.test_views.ListV iewTest.test_validation_errors_end_up_on_lists_page) --------------------------------------------------------------------- [...] AssertionError: False is not true : Couldn't find 'You can't have an empty list item' in the following response
An unexpected failure—it’s actually in the tests for our final view, view_list(). Once again, because we’ve changed the base template, which is used
by all views, we’ve made a change that impacts more places than we intended. Let’s follow our standard pattern, get back to a working state,
and see if we can dig into this a bit.
Get Back to a Working State
Let’s restore the old [% if %} in the template,
so we display errors in both old and new cases:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
{% if form.errors %}
<div class="invalid-feedback">{{ form.errors.text }}</div>
{% endif %}
</form>And add an item to our stack:
A Helper Method for Several Short Tests
Let’s take a look at our tests for both views, particularly the ones that check for invalid inputs:
class NewListTest(TestCase):
[...]
def test_validation_errors_are_sent_back_to_home_page_template(self):
response = self.client.post("/lists/new", data={"text": ""})
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "home.html")
expected_error = html.escape("You can't have an empty list item")
self.assertContains(response, expected_error)
def test_invalid_list_items_arent_saved(self):
self.client.post("/lists/new", data={"text": ""})
self.assertEqual(List.objects.count(), 0)
self.assertEqual(Item.objects.count(), 0)
class ListViewTest(TestCase):
[...]
def test_validation_errors_end_up_on_lists_page(self):
list_ = List.objects.create()
response = self.client.post(
f"/lists/{list_.id}/",
data={"text": ""},
)
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "list.html")
expected_error = html.escape("You can't have an empty list item")
self.assertContains(response, expected_error)I see a few problems here:
-
We’re explicitly checking that validation errors prevent anything from being saved to the database in
NewListTest, but not inListViewTest. -
We’re mixing up the test for the status code, the template, and finding the error in the result.
Let’s be extra meticulous here, and separate out these concerns. Ideally, each test should have one assert. If we used copy-paste, that would start to involve a lot of duplication, so using a couple of helper methods is a good idea here.
Here’s some better tests in NewListTest:
from lists.forms import EMPTY_ITEM_ERROR
[...]
class NewListTest(TestCase):
def test_can_save_a_POST_request(self):
[...]
def test_redirects_after_POST(self):
[...]
def post_invalid_input(self):
return self.client.post("/lists/new", data={"text": ""})
def test_for_invalid_input_nothing_saved_to_db(self):
self.post_invalid_input()
self.assertEqual(Item.objects.count(), 0)
def test_for_invalid_input_renders_list_template(self):
response = self.post_invalid_input()
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "home.html")
def test_for_invalid_input_shows_error_on_page(self):
response = self.post_invalid_input()
self.assertContains(response, html.escape(EMPTY_ITEM_ERROR))By making a little helper function, post_invalid_input(),
we can make three separate tests without duplicating lots of lines of code. We’ve seen this several times now.
It often feels more natural to write view tests as a single,
monolithic block of assertions—the view should do this and this and this,
then return that with this.
But breaking things out into multiple tests is often worthwhile; as we saw in previous chapters, it helps you isolate the exact problem you have when you later accidentally introduce a bug. Helper methods are one of the tools that lower the psychological barrier, by reducing boilerplate and keeping the tests readable.
Let’s do something similar in ListViewTest:
class ListViewTest(TestCase):
def test_uses_list_template(self):
[...]
def test_renders_input_form(self):
[...]
def test_displays_only_items_for_that_list(self):
[...]
def test_can_save_a_POST_request_to_an_existing_list(self):
[...]
def test_POST_redirects_to_list_view(self):
[...]
def post_invalid_input(self):
mylist = List.objects.create()
return self.client.post(f"/lists/{mylist.id}/", data={"text": ""})
def test_for_invalid_input_nothing_saved_to_db(self):
self.post_invalid_input()
self.assertEqual(Item.objects.count(), 0)
def test_for_invalid_input_renders_list_template(self):
response = self.post_invalid_input()
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "list.html")
def test_for_invalid_input_shows_error_on_page(self):
response = self.post_invalid_input()
self.assertContains(response, html.escape(EMPTY_ITEM_ERROR))And let’s rerun all our tests:
$ python src/manage.py test lists Found 21 test(s). [...] OK
Great! We now feel confident that we have a lot of very specific unit tests, which can point us to exactly what goes wrong if we ever make a mistake.
So let’s have another go at using our form for all views,
by fully committing to the {{ form.errors }} in the template:
@@ -18,9 +18,6 @@
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
- {% if error %}
- <div class="invalid-feedback">{{ error }}</div>
- {% endif %}
{% if form.errors %}
<div class="invalid-feedback">{{ form.errors.text }}</div>
{% endif %}And we’ll see that exactly one test is failing:
FAIL: test_for_invalid_input_shows_error_on_page (lists.tests.test_views.ListVi ewTest.test_for_invalid_input_shows_error_on_page) [...] AssertionError: False is not true : Couldn't find 'You can't have an empty list item' in the following response
Using the Form in the Existing Lists View
Let’s try and work step by step towards fully using our form in this final view.
Using the Form to Pass Errors to the Template
At the moment, one test is failing because the view_list() view for existing lists is not populating form.errors in the invalid case. Let’s address just that:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
error = None
form = ItemForm() (2)
if request.method == "POST":
form = ItemForm(data=request.POST) (1)
try:
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
except ValidationError:
error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error} (3)
)| 1 | Let’s add this line, in the method=POST branch,
and instantiate a form using the POST data. |
| 2 | We already had this empty form for the GET case, but our new one will override it. |
| 3 | And it should now drop through to the template here. |
That gets us back to a working state!
Found 21 test(s). [...] OK
Refactoring the View to Use the Form Fully
Now let’s start using the form more fully, and remove some of the manual error handling.
We remove the try/except and replace it with an
if form.is_valid() check, like the one in new_list():
@@ -26,13 +26,11 @@ def view_list(request, list_id):
if request.method == "POST":
form = ItemForm(data=request.POST)
- try:
+ if form.is_valid():
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
- except ValidationError:
- error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error}And the tests still pass:
OK
Next, we no longer need the .full_clean(),
so we can go back to using .objects.create():
@@ -27,9 +27,7 @@ def view_list(request, list_id):
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
- item = Item(text=request.POST["text"], list=our_list)
- item.full_clean()
- item.save()
+ Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)The tests still pass:
OK
Finally, the error variable is always None,
and is no longer needed in the template anyhow:
@@ -21,7 +21,6 @@ def new_list(request):
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
- error = None
form = ItemForm()
if request.method == "POST":
@@ -30,6 +29,4 @@ def view_list(request, list_id):
Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)
- return render(
- request, "list.html", {"list": our_list, "form": form, "error": error}
- )
+ return render(request, "list.html", {"list": our_list, "form": form})And the tests are happy with that!
OK
I think our view is in a pretty good shape now. Here it is in non-diff mode, as a recap:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
form = ItemForm()
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)
return render(request, "list.html", {"list": our_list, "form": form})I think we can give ourselves the satisfaction of doing some crossing-things-out:
Phew!
Hey, it’s been a while, what do our FTs think?
[...] ====================================================================== ERROR: test_cannot_add_empty_list_items (functional_tests.test_list_item_valida tion.ItemValidationTest.test_cannot_add_empty_list_items) --------------------------------------------------------------------- [...] selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: .invalid-feedback; [...] [...] Ran 4 tests in 14.897s FAILED (errors=1)
Oh. All the regression tests are OK, but our validation test seems to be failing—and failing early too! It’s on the first attempt to submit an empty item. What happened?
An Unexpected Benefit: Free Client-Side Validation from HTML5
How shall we find out what’s going on here?
One option is to add the usual time.sleep just before the error in the FTs,
and take a look at what’s happening while they run.
Alternatively, spin up the site manually with manage.py runserver if you prefer.
Either way, you should see something like HTML5 validation says no.
It seems like the browser is preventing the user
from even submitting the input when it’s empty. It’s because Django has added the required attribute to the HTML input
(take another look at our as_p() printouts from earlier if you don’t believe me,
or have a look at the source in DevTools).
This is a feature of HTML5; browsers nowadays will do some validation at the client side if they can, preventing users from even submitting invalid input. That’s actually good news!
But, we were working based on incorrect assumptions about what the user experience was going to be. Let’s change our FT to reflect this new expectation:
class ItemValidationTest(FunctionalTest):
def test_cannot_add_empty_list_items(self):
# Edith goes to the home page and accidentally tries to submit
# an empty list item. She hits Enter on the empty input box
self.browser.get(self.live_server_url)
self.get_item_input_box().send_keys(Keys.ENTER)
# The browser intercepts the request, and does not load the list page
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:invalid") (1)
)
# She starts typing some text for the new item and the error disappears
self.get_item_input_box().send_keys("Purchase milk")
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:valid") (2)
)
# And she can submit it successfully
self.get_item_input_box().send_keys(Keys.ENTER)
self.wait_for_row_in_list_table("1: Purchase milk")
# Perversely, she now decides to submit a second blank list item
self.get_item_input_box().send_keys(Keys.ENTER)
# Again, the browser will not comply
self.wait_for_row_in_list_table("1: Purchase milk")
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:invalid")
)
# And she can make it happy by filling some text in
self.get_item_input_box().send_keys("Make tea")
self.wait_for(
lambda: self.browser.find_element(
By.CSS_SELECTOR,
"#id_text:valid",
)
)
self.get_item_input_box().send_keys(Keys.ENTER)
self.wait_for_row_in_list_table("2: Make tea")| 1 | Instead of checking for our custom error message,
we check using the CSS pseudo-selector :invalid,
which the browser applies to any HTML5 input that has invalid input. |
| 2 | And we check for its converse in the case of valid inputs. |
See how useful and flexible our self.wait_for() function is turning out to be?
Our FT does look quite different from how it started though, doesn’t it? I’m sure that’s raising a lot of questions in your mind right now. Put a pin in them for a moment; I promise we’ll talk. Let’s first see if we’re back to passing tests:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
A Pat on the Back
First, let’s give ourselves a massive pat on the back: we’ve just made a major change to our small app—that input field, with its name and ID, is absolutely critical to making everything work. We’ve touched seven or eight different files, doing a refactor that’s quite involved…this is the kind of thing that, without tests, would seriously worry me. In fact, I might well have decided that it wasn’t worth messing with code that works. But, because we have a full test suite, we can delve around, tidying things up, safe in the knowledge that the tests are there to spot any mistakes we make. It just makes it that much more likely that you’re going to keep refactoring, keep tidying up, keep gardening, keep tending to your code, and keep everything neat and tidy and clean and smooth and precise and concise and functional and good.
And it’s definitely time for a commit:
$ git diff $ git commit -am "use form in all views, back to working state"
But Have We Wasted a Lot of Time?
But what about our custom error message? What about all that effort rendering the form in our HTML template? We’re not even passing those errors from Django to the user if the browser is intercepting the requests before the user even makes them! And our FT isn’t even testing that stuff any more!
Well, you’re quite right. But there are two or three reasons all our time hasn’t been wasted. Firstly, client-side validation isn’t enough to guarantee you’re protected from bad inputs, so you always need the server side as well if you really care about data integrity; using a form is a nice way of encapsulating that logic.
Also, not all browsers fully implement HTML5,[2] so some users might still see our custom error message. And if or when we come to letting users access our data via an API (see Online Appendix: Building a REST API), then our validation messages will come back into use. On top of that, we’ll be able to reuse all our validation and forms code when we do some more advanced validation that can’t be done by HTML5 magic.
But you know, even if all that weren’t true, you can’t be too hard on yourself for occasionally barking up the wrong tree while you’re coding. None of us can see the future, and we should concentrate on finding the right solution rather than the time "wasted" on the wrong solution.
Using the ModelForm’s Own Save Method
There are a couple more things we can do to make our views even simpler. I’ve mentioned that forms are supposed to be able to save data to the database for us. Our case won’t quite work out of the box, because the item needs to know what list to save to. But it’s not hard to fix that!
We start, as always, with a test.
Just to illustrate what the problem is,
let’s see what happens if we just try to call form.save():
def test_form_save_handles_saving_to_a_list(self):
form = ItemForm(data={"text": "do me"})
new_item = form.save()Django isn’t happy, because an item needs to belong to a list:
django.db.utils.IntegrityError: NOT NULL constraint failed: lists_item.list_id
Our solution is to tell the form’s save method what list it should save to:
from lists.models import Item, List
[...]
def test_form_save_handles_saving_to_a_list(self):
mylist = List.objects.create()
form = ItemForm(data={"text": "do me"})
new_item = form.save(for_list=mylist) (1)
self.assertEqual(new_item, Item.objects.get()) (2)
self.assertEqual(new_item.text, "do me")
self.assertEqual(new_item.list, mylist)| 1 | We’ll imagine that the .save() method takes a for_list= argument. |
| 2 | We then make sure that the item is correctly saved to the database, with the right attributes. |
The tests fail as expected, because as usual, it’s still only wishful thinking:
new_item = form.save(for_list=mylist) TypeError: BaseModelForm.save() got an unexpected keyword argument 'for_list'
Here’s how we can implement a custom save method:
class ItemForm(forms.models.ModelForm):
class Meta:
[...]
def save(self, for_list):
self.instance.list = for_list
return super().save()The .instance attribute on a form represents the database object
that is being modified or created.
And I only learned that as I was writing this chapter!
There are other ways of getting this to work,
including manually creating the object yourself,
or using the commit=False argument to save,
but this way seemed neatest.
We’ll explore a different way of making a form "know" what list it’s for
in the next chapter. A quick test run to prove it works:
Ran 22 tests in 0.086s OK
Finally, we can refactor our views. new_list() first:
def new_list(request):
form = ItemForm(data=request.POST)
if form.is_valid():
nulist = List.objects.create()
form.save(for_list=nulist)
return redirect(nulist)
else:
return render(request, "home.html", {"form": form})Rerun the test to check that everything still passes:
Ran 22 tests in 0.086s OK
Then, refactor view_list():
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
form = ItemForm()
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
form.save(for_list=our_list)
return redirect(our_list)
return render(request, "list.html", {"list": our_list, "form": form})We still have full passes:
Ran 22 tests in 0.111s OK
And:
Ran 4 tests in 14.367s OK
Great! Let’s commit our changes:
$ git commit -am "implement custom save method for the form"
Our two views are now looking very much like "normal" Django views:
they take information from a user’s request,
combine it with some custom logic or information from the URL (list_id),
pass it to a form for validation and possible saving,
and then redirect or render a template.
Forms and validation are really important in Django—and in web programming, in general—so let’s try to make a slightly more complicated one in the next chapter, to learn how to prevent duplicate items.
Comments