A common question we often receive about Behave Pro is "How can we publish automated tests results to Behave Pro and Jira from tool X". The short answer to this question is "you can't" but we have a very good reason to why that is. 

Behave Pro and Jira become the central source of truth for Features and Scenarios within projects. Test Automation is enabled by exporting these features and scenarios from Behave Pro within the projects build process e.g. CI process. 

Note: every commercial BDD authoring tool has copied this repository and export model.

Because the projects source code is versioned separately from the Feature and Scenarios its possible to produce passing or failing test results from differing combinations of the source code and scenarios versions. If we were to report test results and publish them in Jira it is hard to know which combination takes precedence over another and a false positive or false negative results could be displayed

One of our company values is "Don't F**k the customer". Because the above situation can provide false information (low probability of this happening but it will happen) to teams we have decided it goes against our company values and have decided that we won't implement this feature until we resolve this false reporting issue.

Sorry, we have taken a ethical stance on the issue.

But everyone else supports it?

Yes they do, but all the other BDD products use the same "export" process and have the same technical flaw as we have described above. The main difference between us and them is we have taken an ethical stance on the issue and won't implement it

You will be pleased to hear we have been working on linking features and scenarios to source control systems. This will resolve the root cause of false test result reporting issue and will implement Test reporting straight away.

A work around

If you are following Continuous Integration, Continuous Delivery/Deployment then you probably know the whole result of the build process is more important than a single test result.

It gives a false impression of the User Story status if the Story displays its own limited set of tests as passing but the whole build is failing but it's hidden from the team. If the CI build is failing then working valuable software hasn't been produced and the User Story shouldn't be considered 'done'.

Jira supports displaying build results on Issues. It will happen automatically if you have linked Bamboo and Jira together but there are plugins for most major CI tools and platforms to do this.

We recommend everyone to setup this feature as the build results is important to understanding if a whole User Story is done and what builds are available for manual testing.

If you are a Bamboo user you might want to check out our plugin BDD Insights for Bamboo. it's free for a limited time.

Did this answer your question?