Sauce Labs Review

PROS: With SauceLabs we had the opportunity to scale our automated tests for web application UI to several browsers so it was difficult because we were forced to implement browser-specific forks in our code, but it was very easy to achieve that with its unified API, instead of setting up several environments on our testing servers. We were able to catch the traffic that our app produces on Sauce Cloud machines for testing, using this nice Sauce Connect feature.

CONS: SauceLabs website has plenty of documentation and you have to search for a bare working example. We could not start our mobile application at SauceLabs because of their spreading of example for mobile tests throughout several pages in their documentation. Lots of parameters of Sauce Connect, Rest API are very poor documented so we wasted lot of time to find out the working code with valid values. Its support sometimes is good, sometimes is bad. Sometimes they transfer me to the Engineering team without knowing when the problem will be resolved. Other thing I dislike is that they often edit by default in case to make them more testable. It brings unexpected behavior to test, for instance, it uses Safari with 3rd party cookies and here is no way to change this, but original feature of Safari is to disable them.

PROS: There are many things that I like about Sauce Labs, but what I like most is the ability to create clean vm for each test, and I am able to take the maintenance of vm out of my daily work. Also, I am able to create various OS and Browser combinations and those are very hard to do manually, so Sauce labs works great. I like the idea to install only one browser instead of installing multiple for running tests. I get my test results with videos and screenshots and everything I need is in one place. And last but not least, the customer support team is excellent, always helpful and responsive, they have helped me with every issue I have faced.

CONS: Oh, I don’t like to use the word “dislike” about this product, because as I said before it works perfectly well. There are things I would like to see improved. When you run single test on Sauce they run fast, but the thing is if you have several in the same time, but, it’s not a big deal. I would like to run my automations (for mobile device) on real devices vs in simulators and emulators. If they could improve the performance and make a connection with real devices there won’t be a need to host own test environment anymore.

PROS: Sauce Labs is smart app that allows me to tackle all testing challenges I have. It offers many testing platforms. I use this software because of the mindset on device, easy to set up, better communication with my account team, pricing and technology. Even though their device list is not as big as the list from other vendor, I don’t have to wait for each device to run on, I can do it whenever I want on the device I want. The other thing I like is that I don’t have to pay for accessing a device and concurrency.

CONS: I have seen that some of my tests are not completed, but I believe they will improve this one. Also, the troubleshooting doesn’t work as it should, I cannot check why some of my tests failed. I have found that the Real Device pool is somewhat limited. I would love to see some things expanded and improved.

PROS: Sauce’s service is straightforward to use and is consolidated with the tests that exist. To add, the customer service is magnificent and I like when they are coordinated with the testing and DevOps industry

CONS: There is nothing much I do not like, except the problem with their manual VMs that are very slow to use and it would be great if they could make their design better.

PROS: Sauce labs offers many platforms for running tests, and support for all great frameworks and it works great with Jenkins. I love its flexibility because it can be used in many ways. It has dedicated UX team and they really care of their customer’s experience and they do everything to improve themselves, their engineers are very responsive and knowledgeable.

CONS: I don’t like that when their infrastructure code is updated no one gets notification about the update. So, I don’t update anything and I face some breaking issues or test failures. So I have to figure out what’s going on and what went wrong and where, this is time wasting process and I end up with sending customer support ticket to the Sauce Labs.

PROS: SauceLabs is fast and other solutions we have tried could not keep with VMs that are always available. Our test was from more than five hours run time on Xamarin to less than 18 minutes by influencing parallel test runners. The best thing is that there is not additional configuration and it uses Selenium Grid so we are sure that our test suites will finish it soon to give a feedback to developers. It is worth the price.

CONS: My biggest issue is that the reporting portal is slow to load. SauceLabs is always improving their product, I have noticed changes at the dashboard but loading videos is slow and there are large lists of test runs.

PROS: When I started with Sauce Labs, I found that their customer support was unresponsive if the fix wasn’t worth it or if you already signed for a contract. But, they changed their behavior and strategies, and they turn into a company that listens its customers and their needs. They created special site for their users, where they get suggestions and help from them in order to become better. After this, my confidence with Sauce Labs has grown, and now they are more responsive and whenever I have some issues they are always there to help. I know they want the best for their customers, and I am very satisfied with the product.

CONS: The only thing I don’t like is that I need to enter my pass and usernames on each site I want to work on. It would be great if they could make an option to choose whether or not I want to login in with pass and username. It would be great if they could find better solution for this issue, because I work/test 10 different sites and I’m wasting my time with entering all the info (10 times).

PROS: What I love about Sauce Labs is the fact that offers free version for testing open source projects (of course with limited resources). It provides great integration with Travis-CI and there are adapters for all types of JS test runners. So I choose SL for my JuiceShop projects, and I am able to get my tests run on multiple browsers and OS combinations. Also, I am provided with full end-to-end test suite that is written on Sauce Labs with Protractor execute.

CONS: Sometimes tunnels don’t come up and tests fail mostly on first run. I think this is because of the free VMs that you get as open source project testing.

PROS: We use Sauce labs few months by now, and all I can say that it is perfect product for site testing. During web interface transition we started using the product and we have great experience. What I want to say is that the testing suits are full with reliable information and we can easily validate across the whole platform and switching between its results and our own. Once we tapped into their API, everything was easy for us. We have found that troubleshooting is less daunting, they have improved their documentation system, and we use its services for mobile iOS and web testing for our projects.

CONS: The adoption of iOS on the system is bit slowly, but I believe they make their best to improve the infrastructure in order to make the updating easier.

PROS: I love the ability to test various browsers and OS combinations. Also, I love the CI integration with Jenkins because this makes an important part of ATDD. I believe that without this integration we wouldn’t be able to include an important part in our development process. This is by far the most important step Sauce has made.

CONS: I don’t know why but sometimes the tests can fail for no reason at all, and then I have to start all over again. This is not big deal but it is frustrating. Also, the inevitability takes more time to spin up a vm and to run some tests on it instead of running the test locally, but I believe that this is necessary so we can get all the benefits that this software offers.