Make test suite text-independent


(goob) #1

A recent PR to make capitalisation consistent throughout the English UI broke a lot of tests, because the strings expected had slightly different capitalisation. I had already considered the benefit of making these tests independent of the text strings to make them more robust to such trivial changes to text.

I see two ways this needs to be done:

  • using I18n.t(“path.to.string”) syntax instead of plain text for strings;
  • using commands such as ‘I submit the form’ instead of ‘I press “Submit”’.

Other types of change may of course become apparent as the work goes on!

Anyway, I’ve pushed some initial work to my fork so the core devs can have a look at it: here’s the commit.

See what you think. Will this be useful work for the project? If so, is my approach the best, or is there a better way to achieve this text-independence?

I’ve started with the cukes because they’re the ones I’m most familiar with, but this can or course be extended to Rspec and Jasmine tests.

I look forward to hearing your views.


Note: This discussion was imported from Loomio. Click here to view the original discussion.


(Jonne Haß) #2

Yes, I think this is the most pragmatic approach.

I’ll leave some comments on your commit.


(Flaburgan) #3

I submit the form

What about pages with multiple forms?


(goob) #4

What about pages with multiple forms?

I meant to do that when it is possible. However, it appears to focus on the form which has been filled out in the previous step, as the tests pass; but I agree, someone who understands the syntax would have to check that this will work on pages with more than one form.


(raven24) #5

strictly speaking steps like “i click ‘submit’” should be considered obsolete scaffolding and better be replaced by steps like “i submit the whatever-form” anyway :wink:
also, to adhere to common testing best-practices all external dependencies should be stubbed, which would include translations in every other test that doesn’t directly test translations.

tl;dr: yes, you would definitely improve test quality with that :slight_smile:


(Jonne Haß) #6

to adhere to common testing best-practices all external dependencies should be stubbed

Our cucumber tests serve as integration tests, so that doesn’t apply there.


(goob) #7

OK; thanks, both. I’ll start on this later in the week if there are no objections from other core members.

tl;dr: yay!


(raven24) #8

@jhass you’re right … I am still over-stimulated from my software quality course last semester :wink:


(goob) #9

Just shows that there is a thing as too much quality… :slight_smile:


(goob) #10

I’ve got most steps working text-independently, but have hit two snags:

  1. The ‘should see the text for’ steps don’t interpret variables in the strings. For example, desktop/follows_tags.feature:18 looks for the exact string “Follow #%{tag}” and won’t accept “Follow #boss”. I don’t know the syntax well enough to have a hope of solving this. Is there a simple way of accounting for variables in the strings, or should I revert these steps to how they were before?
  2. For some reason, whenever I try to rebase against develop, only the first commit to this branch is included, so I haven’t been able to rebase. I’ve tried everything I can think of, but with no success. You can see on the commit history that I’ve been making little commits as I’ve gone, assuming that I’d be able to rebase into one commit once I was finished. Any help you can give would be much appreciated!

(Jonne Haß) #11

I think leave them at the hardcoded text for now and lets revisit when you opened a PR, where it’ll be much much easier to discuss solutions.

As for rebasing, I’d need to look over your shoulder to see what’s going wrong :confused:


(goob) #12

Sure, will do. PR to follow soon.

I guess I’ll have to copy the files, wipe the branch and add them clean. No hassle (well, not much).

:slight_smile: