great post

I can't believe here we are in 2018 and this hasn't been done yet

My thoughts based on the previous comment : 2017-09-30 below, I'd like to see some kind of new component of the system. Something like Project Steps that can be attached to a project to "Verify" either at the project level or the Step level.

* Check 200 for PayLoad
* Post {data} to Address confirm ReponseCode
* Run query .. expect N Records (GIGO principal on the query)
* Check for File

Then having the same kind of thing as step templates when I can make my own and have them either part of our Octopus server for others to consume or Community templates.

Having the ability to integrate testing results directly into the Octopus UI would be powerful and make octopus a truly enterprise grade deployment tool. Writing powershell or C# testing scripts is not ideal and adds complexity to the whole release process.

Any movement on this?

I'm considering using Octopus Deploy as an integration testing tool. The pipeline is the following: a build is taken from TeamCity, Octopus deploys it on a number of VMs, the integration tests are executed, depending on the execution results, the build is tagged in TeamCity as tested or not.

As I see it now, the really missing piece of the puzzle is showing test results in the deployment log. We can attach a file, but something similar to test results view in TeamCity would make OD very integration/smoke tests friendly.

Smoke tests don't really sit within the scope of Octopus - however I've got this working in conjunction with TeamCity. TeamCity is responsible for taking the source code, running unit tests, packaging and sending build artifacts to Octopus. Octopus then deploys the website, and after deployment runs a powershell script that triggers another TeamCity build. This new TC build is responsible for running the test script (in my case it's selenium based functional testing), and returns a non-zero code if the tests fail. This means that the deployment is 'paused' while tests are executed. In the future I'm going to look into TeamCity generating a useful test report which can then be automatically uploaded as an Octopus deployment artifact.

Any build specific stuff (such as changesets, etc) are done in TeamCity just before packaging, when TC will generate a PostDeploy.ps1 script that echos octopus variables I can access during the deployment.

It's not a perfect solution but it does work, and means fully automated tests and deployments.

Powershell Script:
https://gist.github.com/munkyjunky/44fba7ab7b36c4dbeeab95dec2c125a3

There's already some pretty cool scripting capabilities utilizing powershell and C#. A build step could be defined to run the NUnit xml results through third party tools like NUnit Orange, so the transformation of test result output is probably something beyond the scope of Octopus. Though, the ability to view those results in the dashboard would be very efficient. Maybe there's already a way to add a clickable link to a build step, that would point to wherever the test results were hosted? Another option would be to allow the test results to be uploaded and add some navigation to the uploaded page to the dashboard, but this introduces a security issue where one might generate html pages to hijack the system admin's account. Yet another option would be to allow for custom results parsers like Jenkins or Hudson, the community could build parsing solutions and Octopus could host them. This last one would be the most complicated because of licensing issues and development time, but it would also be the most awesome... We just create a step to run the test and output the results where Octopus can find them, then define the parser to transform results into some beautiful html page that the task results can point to and see it right there in the dashboard.

The solution I'm using now is to create my own executable that runs the NUnit tests, passes the results through NUnit Orange, and puts both files out to a directory being hosted by a simple IIS site. This is cool and all, but it's not integrated with the dashboard. People have to see the deploy fail, then navigate to the internal site and find the results themselves. They seem pretty excited about the idea of being able to just click a link and find the results right in front of them instead. So I'm here. :D

A blog detailing how to run Nunit and incorporate the results in the dasboard would be great.
Also failing the pipeline if the test dont run, so a promotion is not possible.

I agree, i would appreciate to see an example of it.

I would love to see more blog posts on how this can be achieved as this is something that I would like to do as well, but I have not been able to put the pieces together yet. Is it possible to share the configuration, process steps that you use internally somewhere so we can see how to do it. Does anyone have any links that they can share?

Hi - has there been any more official blog posts/documentation added to show how integration testing is properly done using octopus deploy ? We are using OD v2.4.7.85

In our company we use Lab Manager (MTM).
We have a CI build that creates a Nuget package and then call Octopus to create a release in our "Staging" environment.
At the end of the build, MTM runs the tests on top of the release and then we update the build output with the test results. Quite easy to achieve with Octopus + TFS + MTM

1. More blog posts/documentation is ALWAYS welcome. :)
2. Attaching the test results would be fine in my opinion
3. Badges, or some UI indicator of Pass/Fail is really the key ingredient here. Maybe in the dashboard, color the deployment as yellow to indicate that it was deployed, but tests failed?

We need an easy way to get the 'badges' showing success/failure from the result of PS scripts or nunit, etc. Also, once implemented, a good blog post would be helpful.

Yeah exactly, I think a means to integrate the xml with the browser would be ideal. Reading the xml is kind of a drag, as is using a viewer to check out the results. Your last paragraph basically hit it dead on the head, for each deployment having a list, whether that's badges or something else, of what tests passed (and which failed) is what I was seeking when I initially voted on the thread. Thanks for the response!

This would be great. I think some sort of basic support for websites should be included. Just give OD a url and it hits it looks for a 200 result. That would be a place to start. Full UI testing would be more like what you specified above, but I would like just a built in 'sanity check'.

We run our selenium ui tests using mstest.exe as part of deploy.ps1, we then fail the deployment if the tests fail. Finaly we hoover up the log files so you can see them in the octopus ui under output.