What is the best CI/CD pipeline to integrate in mobile app development?

Question

I'm currently working on a trade study with the following objective:

Implement an automated CI/CD Pipeline to perform product deployments in the most optimal time possible that is free for mobile app development. *Coding in React-Native*

The CI/CD Platforms I am currently looking at are:

Travis-CI
Circle-CI
Jenkins
Gitlab-CI
AWS CodePipeline
Google Cloud Build
Azure Pipeline

And my trade factors are: 
Cost (Is there some kind of plan for free usage.)
Security Performance Speed ( Time at which it takes to ship out a new build.)
Hosting (Yes/No)
Flexibility 
Maintainability
Distribution
Reliability
Documentation
Customer Success Capability

I would absolutely appreciate any feedback you all have to offer, especially since I am new to mobile app development. Anything you know about these various CI/CD platforms in the Trade Factors listed or if you believe there is a Trade Factor that I should include or exclude.

If this is successful I will "pay it forward" by putting together an article with all my findings so anybody else stuck in the same spot can derive from that. Thank you all ahead of time!

Ruslan · Accepted Answer

It all depends on your abilities and knowledge of the code. Check out the Nwicode CMS platform, which is open source (which gives you an opportunity in product extensibility) and a large number of ready-made features, for quick creation without coding.

Andrew Kirima · Answer

@alejandro_cantarero1 Thank you for that Alejandro! Probably the most intuitive answer yet. I have a meeting/demo with GitLab and CircleCI coming up so I think I'll get a bit more understanding. Also, I am new to DevOps, I've always known of it but this my first hands-on experience so I'm trying to play it carefully. However, I am starting to question myself a bit. Do you think implementing a CI/CD pipeline pre-launch is recommended? What questions should I be asking myself to figure out when is the right time to implement one?

We haven't launched a single product yet, so this will be for our first one as a startup. I'm hoping to get the MVP out by December or January at the latest. Finally, just wanted to let you know that as a startup our only option is a free tier if we find it essential to implement one pre-launch.

Alejandro Cantarero · Answer

@andrew_kirima1 To be honest, I'm a strong believer in it is never too early to set up CI/CD (assuming you can control the costs / stay in the free tier band).

In particular, I've always found that without a good test suite it is really easy to break things.  And even if you have the test suite, without enforcement when code is merged into your projects it is easy for things to break.

In particular to our mobile products as a startup, there is never enough time to do a full set of user tests around the entire product when the current change is small in scope.  Usually we just test that the new feature is working correctly, and rely on the test suite to tell us something broke elsewhere.

We've definitely caught many bugs this way.

If your mobile app relies on a server, CI/CD there is also super helpful.

Our CD on our server has allowed us to make several deploys in a day when we are iterating quickly and making changes for a new feature in our mobile products.

Alejandro Cantarero · Answer

@andrew_kirima1  I'd take a careful look at their free tier to see if it does enough for you.  That can vary a lot with the SaaS tools.

Once you start growing another important question is do you want to be testing on multiple mobile OS versions / devices.

That's one of the downsides of GitHub Actions, they only support 1 version of Xcode and iOS simulators at a time, so there's no way to do a really broad set of tests.

Also, how big of a community is there around the tool.  With the bigger tools you often don't have to reinvent the wheel as some likely already has a solution to what you are trying to do that you can either directly use or modify.

A concrete example from Github again (since that's what we use), is they have a marketplace of actions available, so if you want some stage of your CI/CD process to send a message to slack, you don't have to figure that out yourself, you can just grab an action off the marketplace.

Alejandro Cantarero · Answer

We have built two CI pipelines for our iOS and Android SDKs.   Here’s some notes on what we’ve found worked for us.

The first pipeline was with CircleCI and then we moved to GitHub Actions.  The main reason for making the move was that our code repos were are all Github and (at the time) Actions had a very generous free tier.  They have since cut their free tier usage amounts by about 70%.

I’d personally go with a SaaS / hosted solution over setting up your own.  We previously had a Jenkins server and it mostly was great, but periodically we’d have some issue and have to spend a few days fixing it.

We found both CircleCI and Github Actions to be very reliable and maintainable.  Generally very little practical difference for us other than cost.

On both platforms build minutes for iOS are much more expensive then for Android (linux images vs MacOS images).

CircleCI does not include MacOS images in their free plan and Github does.

If keeping costs down is important, one thing we’ve done is limit the devices and OS versions we test on during development.  We then do a more broad test on the final build when we are ready to merge to our main branch.

The one annoying thing with Github is they don’t keep around old OS versions of their simulators, so we very regularly have builds fail because Github has removed the simulators we were using.  They don’t seem to have a good method for notifying about these changes.

This also prevents you from testing on older OS versions in the simulators, so if that's important you'd want to use CircleCI or some other solution.

Andrew Kirima · Answer

@alejandro_cantarero1 Okay, then I'm sold that I'm doing the right thing by looking into this right now. However, would you have some feedback on the type of questions I should be asking myself to make sure I choose the right CI/CD pipeline? Also, what should I be aware of to know that chose the right/wrong tool?

Thanks so far for your feedback. You're the first person to really give me direct feedback to make decisions instead of getting thrown in loops.

Andrew Kirima · Answer

@andrew_kirima1 @alejandro_cantarero1 so would you say the following trade factors are legit:

1) Pricing: What’s the cost model? Depending on the number of builds you will be doing certain tools will charge you differently. On larger projects, you may end up running builds many times a day. Some vendors may impose limits on this or may charge based on this.  Do you need to be building concurrently? If you have more than 1 team in the development, then they may all want access to the build toolchain at the same time. This means you may need more infrastructure or more resources from a SaaS vendor. How many users do you need? Many vendors will charge you more the more users you have. Is there a free tier if you don’t have a budget for this?

2) Reliability: Is your tool Open-Source or a Proprietary tool? Open source is free to use but if it’s not a popular one then it could lose support. However, a proprietary tool has constant support, but it cost more. How well does it work? What does everyone else say about it?

3) Usability & Learning Curve: How easy is it to use and if you have questions is there a big enough community so you can get support?  How helpful are they?

4)Flexibility: Does it integrate well with our existing setup? How easy is it to integrate What other features does it provide? What can it deploy to? Cloud, On-Premises, or Saas?

5)Security: How secure is it? How often does it security fail? If you’re using a tool that runs on a virtual infrastructure, then you need to consider the data and access keys. It may need access to our SSH keys in our coding repository and credentials from our Github repo.

Andrew Kirima · Answer

@ruslan77 Thank you for taking the time to respond. But we are already programming the app with react-native. It's a fintech app that will be doing somethings that other apps out there haven't done, or at least successfully yet so it needs to be coded. Codeless isn't an option because we need to be able to scale and use a BAAS on the backend.

What is the best CI/CD pipeline to integrate in mobile app development?

Replies