# Promises and Challenges of Microservices: an Exploratory Study --- ## Overall Study Design Our study applied a mixed-method approach: first, we conducted a qualitative semi-structured interview study to identify challenges and lessons learned in microservice-based development. We then distributed an online survey to enrich and validate (or not) the interview findings using collected quantitative data. We synthesized the findings from both the interview and survey studies, forming the final results of our study.  --- ## Interview Study --- ### Data Analysis Process For open coding, two of the authors of the paper independently read the transcripts line by line and identified concepts - _key ideas contained in data_. When looking for concepts, we searched for the best phrase that describes conceptually what we believe is indicated by the raw data. On a weekly basis, all the authors met to discuss the identified concepts and to refine and merge them if needed. That was done using card sorting: each card represented a quote labeled with the concepts, and we grouped the quotes and refined the concepts, as needed.  --- ### Categories Identified in Interview Study  --- ## Survey Study --- ### Survey Questions The survey consisted of 13 groups of questions and a total of 38 questions of mixed types. Most of the questions are multiple-choice, with an additional field to provide extra information. Several open-ended questions were designed to collect information beyond that observed in the interviews. In addition to the microservice background and demographic information, we asked questions related to the ten categories identified in the interview study. In each question, participants had a chance to either agree or disagree with the statements made by the interviewees, identify additional solutions to the identified problems, or report on additional best practices and challenges. We also asked participants to distinguish between the solution they apply in practice (what they are currently _doing_) and their perception (what they _think_ they should do). Here is the list of survey questions: --- Q1: How long have you been developing microservices? [Number entry] Q2: In your most significant microservice-based development project, how long has the team been developing microservices? [Number entry] Q3: How many members are there in the team? [Number entry] Q4: How many microservices does the team develop and maintain? [Numeric range choices] * < 6 * 6-10 * 11-20 * 21-30 * 31-50 * 51-100 * \>100 Q5: How many microservices do you typically work on? [Numeric range choices] * < 6 * 6-10 * 11-20 * 21-30 * 31-50 * 51-100 * \>100 Q6: Are you working on commercial or open source project? [Single choice] * Commercial * Open source > Robust logging and monitoring systems are critical for efficient microservice-based development. Q7: To what extent do you agree with the above statement in general? [Single choice] * Agree. These systems should be set up as early as possible. * Agree. These systems should be set up later during the project. * Disagree. I do not think these systems are necessary. * Other: [Text entry] Q8: In the context of your project, when did you set up logging and monitoring? [Single choice] * As early as possible. * Later during the project. * We did not set them up yet. * Other: [Text entry] Q9: In the context of your project, what information is being logged? [Multiple choice] * Up/down status of microservices. * The number of requests each microservice receives per a certain period. * The input/output of each request. * The request and response time for each call. * Resource consumption of each microservice, such as CPU, memory, and disk. * Other: [Text entry] Q10: In the context of your project, what is the logged information used for? [Multiple choice] * Troubleshooting. * Creating reports for our clients (e.g., availability and latency in SLA reports). * Dashboarding and data visualization (e.g., to plot graphs with response time). * Proactive anomaly detection (e.g., software that analyzes the logged data and generates alerts, when needed). * Other: [Text entry] > Distributed tracing is critical for microservice-based application development and maintenance. Q11: To what extent do you agree with the above statement in general? [Single choice] * Agree. Distributed tracing should be set up as early as possible. * Agree. Distributed tracing should be set up later during the project. * Disagree. I do not think distributed tracing is necessary. * Other: [Text entry] Q12: In the context of your project, when did you set up distributed tracing? [Single choice] * As early as possible. * Later during the project. * We did not set it up yet. * Other: [Text entry] > Automating the microservice setup process (e.g., automatically generating a microservice skeleton and plugging it into the entire product) is critical to save development time and costs. Q13: To what extent do you agree with the above statement in general? [Single choice] * Agree. Such a process should be set up as early as possible. * Agree. Such a process should be set up later during the project. * Disagree. I do not think such a process is necessary. * Other: [Text entry] Q14: In the context of your project, when did you automate the microservice setup? [Single choice] * As early as possible. * Later during the project. * We did not automate it yet. * Other: [Text entry] Q15: In the context of your project, how did you define the granularity of microservices? [Multiple choice] * By business capabilities (i.e., grouping code that performs the same high-level functionality). * By data access (i.e., grouping code that accesses the same data). * By dependencies (i.e., grouping code that depends on a similar set of other sub-systems). * By delivery lifecycle (i.e., grouping code that has to be released together). * By team structure (i.e., grouping code that is developed by the same team). * By resource consumption (i.e., to ensure that the split into microservices does not result in excessive resource consumption due to containerization, etc.) * Other: [Text entry] > Each microservice can be developed using its own programming language. Q16: To what extent do you agree with the above statement in general? [Single choice] * Agree. Always use the most appropriate language for each microservice. * Disagree. A company should restrict to use a specific set/number of languages (e.g., to facilitate an automated microservice setup process). * Other: [Text entry] Q17: In the context of your project, do you regulate the use of programming languages? [Single choice] * No. We always use the most appropriate language for each microservice. * Yes. We have a restricted number / set of languages to use. * Other: [Text entry] > Each microservice should be owned by a specific person or team. Q18: To what extent do you agree with the above statement in general? [Single choice] * Agree. * Disagree. A company should not define owners for microservices. * Other: [Text entry] Q19: In the context of your project, do you identify microservice owners? [Single choice] * Yes. We define a clear owner for each microservice * No. We do not define owners for microservices. * Other: [Text entry] Q20: If you specify service owners, what are their responsibilities? [Multiple choice] * Troubleshoot their microservice * Fix bugs in their microservice * Decide on the design and architecture of their microservice * Implement new feature requests * Other: [Text entry] > In microservice-based development, some code is shared by multiple microservices, e.g., the code of authentication, logging, and monitoring. Q21: How should one manage such common code? [Single choice] * As a standalone library that is used by other microservices. Have one version of the library that is used in all microservices. * As a standalone library that is used by other microservices. Allow different microservices to use different versions of a shared library. * As a standalone microservice. * As a standalone microservice that runs as a sidecar. * Other: [Text entry] Q22: In the context of your project, how do you manage common code? [Single choice] * As a standalone library that is used by other microservices. We have one version of the library that is used in all microservices. * As a standalone library that is used by other microservices. We allow different microservices to use different versions of a shared library. * As a standalone microservice. * As a standalone microservice that runs as a sidecar. * We do not have any common code. * Other: [Text entry] > Different customers might get a different variant of the developed product. Q23: How should one implement such variants? [Single choice] * Use feature flags/toggles (run-time decision) * Use role-based access: we check if the client has access to the called API (run-time decision) * Use separate deployments: independent modules are combined for each deployment as needed (build-time decision) * Other: [Text entry] Q24: In the context of your project, how do you implement variants? [Single choice] * Using feature flags/toggles (run-time decision) * Using role-based access: we check if the client has access to the called API (run-time decision) * Using separate deployments: independent modules are combined for each deployment as needed (build-time decision) * N/A: we do not have variants. * Other: [Text entry] Q25: In the context of your project, how do you manage API changes? [Multiple choice] * Via direct API calls * Via a proxy, such as API gateway * Via a client library, such as Swagger * Via message-based communication, such as RabbitMQ * Other: [Text entry] > There are currently more than 200 free and commercial tools in the Cloud Native Interactive Landscape website for supporting microservice-based engineering. Moreover, new tools are frequently developed and become available. Q26: In the context of your project, are you actively evaluating new tools or plan to do so in the future? [Single choice] * We are happy with the tools that we have and rarely look for opportunities to replace them. * We often evaluate newer tools and extend our toolset when better tools become available. * We plan to evaluate newer tools but have not done that yet. * Other: [Text entry] Q27: What is the next big challenge for microservices? [Text entry] Q28: Any additional comments / suggestions you want to share with us? [Text entry] Q29: Did you participate in our earlier interview study? [Yes/no option] * Yes * No Q30-Q37: The following questions are intended to collect demographic information about the survey participant. The questions are fully optional but your answers will be highly appreciated. 1)How long have you been working in industry? [Number entry] 2)Which company do you work at? [Text entry] 3)How long have you been working at the company? [Number entry] 4)What is your job title? [Text entry] 5)What type of application are you developing? [Text entry] 6)What is your age? [Number entry] 7)What is your gender? [Single choice] * Male * Female * Prefer not to say * Other 8)What is the highest degree that you have received? [Single choice] * No degree * Undergraduate level degree (e.g., Bachelor's) * Graduate level degree (e.g., Master's, PhD) * Other Q38: If you would like to receive the results of the study, please fill out your contact information. This information will only be used to contact you with the results. It will not be associated with your other input during data aggregation. Name : [Text entry] Email : [Text entry] Thank you very much for taking the time filling in the survey, we sincerely appreciate your help! ----- ### Survey Results We received a total of 185 responses; 61 of them were complete (completion rate: 33%) and 37 satisfied the same participant selection criteria we applied for interview study: at least two years of microservice development experience. --- ## Mapping Concepts to Quotes Concepts that discuss problems, practices, or concerns around same topics were grouped together to form categories. Our mapping of number of quotes -> concepts -> categories -> themes is below. We only include concepts that were mentioned by at least three interview or survey participants. In the mindmap, concepts reaching the threshold of three participants after the analysis of the survey results are marked in blue.  Digital version can be downloaded [here](https://osf.io/eb96u/).