# The Impact of AI on the Future of Security Questionnaires What 300,000+ security questionnaire questions answered by Conveyor’s AI tell us about the future of security reviews A 2024 Predictions Report by ## Every day, infosec and presales teams lament one thing... Will we be completing questionnaires until the end of time? Security questionnaires have likely been around since the 70s and 80s and in their more “modern” form, since the early 2000s (the years of Myspace, iPods, and more...) So naturally, with all the advances in LLMs today, everyone is wondering... Is AI the key to stopping the questionnaire madness? By analyzing the activity in Conveyor’s AI trust platform in 2023, we try to predict the future of security questionnaires... ## Security Document Downloads - 0K+ Of F500 conducted reviews through Conveyor - 0% Security Questions ANSWERED by CONVEYOR's AI - 0k+ Predictions Summary ### In the Report See data insights + opinions from experts in infosec & presales ### Prediction 1 ## No more question/answer knowledge bases to maintain. Not only will AI use various source materials like a knowledge base, documents, and webpages to generate answers, it will also maintain that data for you. ### SURVEY RESULTS: HEAR FROM THE EXPERTS #### We asked infosec and presales teams: "On a scale of 1-5 (5 being very painful), how painful do you find maintaining your knowledge base?" - Hot Take #1: “I’m gonna say that’s a 4. It is VERY painful.” - Hot Take #2: “A 3. Even if software makes it easier, it’s still overall a horrible task.” - Hot Take #3: “I’ll say maybe a 3, but that’s just the nature of maintenance.” ### What did 2023 tell us? Based on the 300,000 questionnaire answers processed in Conveyor, here’s what we know... #### Today, you need 500-1,000 question and answer pairs in your knowledge base for the highest precision and coverage by AI. #### **If you think your knowledge base is difficult to manage, you’re not alone.** Most companies struggle with maintaining an “effective” knowledge base. - Useless Questions: 0% of a KB on avg is likely filled with “completely” useless questions (never used) - Mostly Useless Questions: 0% of a KB on avg is filled with “mostly” useless questions (used 1-5x in a year) #### **Of the 100 questionnaire topic areas, these were the top 10 most popular topics in 2023** 1. Authentication - Password and MFA 2. Penetration Testing - including summaries 3. Encryption - In-transit, at-rest, and key management 4. Disaster Recovery and Business Continuity 5. Change Management 6. Backups and Retention 7. On-prem vs Cloud hosting 8. Logical Data Separation 9. Critical Third Party Vendors 10. SDLC #### **And the one question you’ll want to keep updated if your company has introduced AI in your software** ## "Does your model train on our data?" ### Prediction 2 ## Trust centers aren’t the final answer… just part of the answer. There will always be a subset of customers who won’t self-serve and expect a “concierge” type service from your team for security reviews. ### What did 2023 tell us? Based on the 100k security document downloads in Conveyor, here’s what we know... #### Even if you have perfect information in your trust center, many customers are likely still going to send a questionnaire. - 0% of customers still sent a questionnaire even after accessing information in the trust center #### Preventing questionnaires depend on the factors below, leverage of the customer (size of deal), maturity of their security organization, and willingness to self-serve. **Will a trust center deflect a questionnaire?** #### A trust center probably won’t deflect a security questionnaire if you’re selling into these industries Conveyor answered the most questionnaires for companies in these industries in 2023. 1. IT Services 2. Financial Services 3. Banking 4. Retail 5. Hospitality & Health Care ### Prediction 3 ## Both parties rely on AI to finish the security review… but only if AI is good enough. As AI accuracy increases, presales teams will start to own the entire customer security review process. ### SURVEY RESULTS: HEAR FROM THE EXPERTS #### **We asked infosec and presales teams:** **"What is the future of security questionnaires?"** - Hot Take #1: “In my pipedream, it will be phased out because more and more companies are leaning toward automating and exposing as much information for self-serve as possible. However, I suspect that with AI becoming more prevalent that companies will feel more empowered to send large questionnaires since they won't feel like they are wasting anyone's time. Maybe when it comes back, they'll have their AI review it and pull out red flags. Ultimately, it will just be robots talking to each other.” - Hot Take #2: “It will be a simple prompt UI for customers - like extremely minimalist and delightful. One-click form completion.” ### What did 2023 tell us? Based on the 300,000 questionnaire answers processed in Conveyor, here’s what we know... #### AI has proven itself to be pretty good as the “tireless, infosec analyst” this year with high rates of precision in question answering. Conveyor's most successful customers were able to achieve on average: - **92% Questionnaire Answer Rate** - **0% Questionnaire Precision** #### AI was able to help teams significantly reduce time spent on security questionnaires **Survey results** If it was possible, would you want your smaller customers to self-serve answers to their questionnaires using AI, such that the only questionnaires you do by hand are for larger or high-value customers? Hot Take #1: “No. Despite gains in AI accuracy, I do not see it reaching a point where we would rely on it completely. We would not want an incorrect response being provided to any customer, large or small, given there could be commercial or regulatory implications to providing incorrect information.” - Hot Take #2: “Yes. In my role, nothing feels like a bigger waste of time than completing/reviewing a questionnaire for a small opportunity.” ### Prediction 4 ## Companies continue sending custom, increasingly detailed questionnaires, but AI helps teams by generating answers instantly ### SURVEY RESULTS: HEAR FROM THE EXPERTS #### **We asked infosec and presales teams:** **"What is the future of security questionnaires?"** - Hot Take #1: “Even more questionnaires with even more requests for details confirming the answers are implemented.” - Hot Take #2: “It still helps to have them because certifications can't cover all and they are out-of-date (agreed on, approved) by default ... so knowing specifics relating to the use case is needed.” ### What did 2023 tell us? From the 300,000+ questions processed by Conveyor’s AI, here’s what we know... #### Custom questionnaires dominate yet again and standardized questionnaires are becoming less common. #### The teams sending questionnaires favor excel over the other file types. #### You can also expect more portal-based questionnaires from these usual suspects. #### What about number of questions? The average questionnaire is 80 questions give or take, but if you sell to industries like telecommunications or legal, you can expect significantly more. ### Avg. Questionnaire Length - 0 Questions - Telecom Industry: 0% Above Average - Law Practice: 0% Above Average We think the ultimate future is... # Custom questions remain and AI does the grunt work Questions about your security posture likely won't go away anytime soon. Although we're not at full automation yet, here are some practical tips to use when automating your customer security reviews today: - Analyze your "one source of truth" - Get sales buy in for a trust center - Delegate to presales/sales teams - Thoroughly evaluate AI solutions