Products: filter questions

Aaron Kirk 23 August 2024 07:07

Filter questions are AI prompts designed by the product builder to convert qualitative inputs into quantitative outputs, providing definitive responses. By asking a closed-ended question with a definitive outcome, documents can be examined and grouped together.

Jylo has been designed with system prompts built into the platform so that minimal prompt engineering is required to get the model to return results in a consistent format. Instead, product builders should focus their efforts on refining their questions so that they are as clear and concise as possible and allow the platform to do the rest.

When designing a filter question, product engineers should familiarise themselves with the necessary output requirements of the answer, and word their prompts in such a way that the model will answer without ambiguity

Filter type	Definition	Output requirements
Yes/No	Questions with a binary yes/no answer	Closed question with no other output possibilities beyond a ‘yes’ or a ‘no’. If the model cannot determine a yes/no answer, will respond ‘N/A’
List of Values	Questions with a categorical answer, employed when the response categories the model should reply with are known	Closed questions with a categorical response. If known in advance, categories should be included in the prompt itself so that the model will select only those possible answers given to it by the product builder If not known, Jylo will run a post-processing prompt on all of the responses returned by the dataset in order to group responses together into as few variations as possible, in order to reduce the number of response filters and assist with drilling down into response. Original responses will also be shown
Range of Values	Questions with a numeric answer, displayed as a range of values in a slider.	Closed questions returning a value in number format only. Product builders should ensure that their prompt does not include formatting requests such as “answer in USD” as non-alphanumeric outputs cannot be represented as a value in the slider

Example Yes / No Filter

Label	Red Flag Content
Description	(TBC)
Question	Does this email contain any content or language that is inflammatory, inappropriate, unprofessional, bullying or harassing in nature?
Dataset	60 x simulated email documents in example email dataset with certain ‘red flag’ emails

Output

12	Yes
48	No

Notes

The prompt has been constructed in a way that there is a binary outcome; either the email content contains a red flag, or it does not. Behind the prompt, the model (ChatGPT 4.0-Turbo) is applying its reasoning ability to read each email and determine whether it meets the standard or does not.

Screenshot

Example List of values filter

Label	Message Category
Description	(TBC)
Question	I am reviewing emails for compliance purposes. Categorise this message based on the following definitions. Response categories: - External Communications - Financial - Internal Communications - IT and Security - Legal and Compliance - Miscellaneous - Operations and Logistics - Personal Correspondence - Social and Cultural
Dataset	60 x simulated email documents in example email dataset with certain ‘red flag’ emails

Output

16	External Communications
0	Financial
31	Internal Communications
0	IT and Security
1	Legal and Compliance
0	Miscellaneous
1	Operations and Logistics
10	Personal Correspondence
1	Social and Cultural

Notes

The use case involves e-Discovery, i.e. sifting through a dataset to find specific categories of information. The simulated dataset represents multiple emails among individuals who are the subject of an investigation. Because the dataset concerns email correspondence, the information categories can be predicted, and a list of response categories can be added in to the prompt itself.

Screenshot

Example Range of values filter

Label	Ethical Breach
Description	(TBC)
Question	Evaluate the content of this document and rate it on a scale of 1-10 for potential ethical breaches. Use the following scale for your rating: 1 – No evidence of unethical behaviour; completely innocuous. 5 - Moderate evidence of unethical practices but not conclusive or minor infractions. 10 - Clear and serious breach of ethical standards in product development. Rate the document based on the severity of any ethical issues identified, considering factors like the evidence of intent, the extent of the violations, and the potential impact on public health
Dataset	60 x simulated email documents in example email dataset with certain ‘red flag’ emails

Output

1	56 x emails
2	3 x emails
9	1 x email

Notes

In this example, no definition of what constitutes an ethical breach has been provided; the selected model (ChatGPT 4.0-Turbo) has utilised its own understanding of the underlying concept and made a determination of the content provided before assigning a value for the breach.

The simulated data has been designed in such a way that the specific text related to ethical breach is nuanced and subtle; because of the model’s ability to read and understand natural language, this was flagged as a 2 / 10.