tasks.md
August 6, 2024 ยท View on GitHub
| Task Name | Description |
|---|---|
| Analogical Problem Solving | Using analogies to solve problems by applying solutions from one domain to another similar situation. |
| Analogy Creation | Creating analogies to compare different concepts, situations, or objects. |
| Analyzing Cultural Differences | Comparing and contrasting different cultural practices, beliefs, or norms. |
| Analyzing Decision Making Processes | Examining and evaluating decision-making processes, including steps, factors, and reasoning. |
| Analyzing Decision Trees | Interpreting and evaluating decision trees to understand decision-making processes. |
| Analyzing Historical Counterfactuals | Considering alternative outcomes of historical events by changing key factors or decisions. |
| Analyzing Rhetorical Strategies | Identifying and evaluating rhetorical strategies used in a given text or speech. |
| Assessing Risk And Uncertainty | Evaluating scenarios with incomplete information to determine potential risks and uncertainties. |
| Bias Detection | Identifying and explaining biases present in given scenarios, statements, or data. |
| Bias Mitigation | Constructing a statement as close to the ground truth as possible, given a statement and the bias that led to it. |
| Calculating Probabilities | Calculating probabilities for various scenarios, including simple and compound events. |
| Categorizing Information Into Hierarchies | Organizing information into hierarchical structures based on relationships or logical criteria. |
| Causal Chain Analysis | Identifying and analyzing a sequence of events or factors that lead to a specific outcome. |
| Completing Analogies | Completing analogies to evaluate the ability to recognize relationships between pairs of words and apply them to new contexts. |
| Constructing Valid Arguments | Creating logically sound arguments to support a given conclusion or claim. |
| Counterfactual Analysis | Examining hypothetical scenarios that are contrary to what actually happened. |
| Critical Factor Identification in Theory of Mind | Understanding that similar agents may act differently, or different agents may act similarly, based on a third factor. |
| Critiquing Argument Structures | Analyzing and evaluating the structure, logic, and effectiveness of arguments. |
| Curry's Paradox | Examining a specific form of semantic paradox that seemingly proves arbitrary claims through self-reference and logical implication. |
| Deciphering Ambiguous Instructions | Interpreting and clarifying unclear or incomplete instructions. |
| Deconstructing Complex Systems | Breaking down complex systems into their constituent parts and explaining how these parts interact. |
| Deconstructing Metaphors | Analyzing and explaining the meaning behind metaphors. |
| Deducing Motives From Actions | Analyzing actions of individuals or groups to infer their underlying motivations or intentions. |
| Deducing Rules From Examples | Analyzing examples to infer the underlying rule or pattern. |
| Deductive Logic Puzzles | Solving deductive logic puzzles to evaluate the ability to use given information, make logical inferences, and arrive at a correct conclusion. |
| Describing Spatial Relationships | Accurately describing the relative positions and orientations of objects in space. |
| Detecting Sarcasm And Irony | Identifying and explaining instances of sarcasm or irony in given statements or scenarios. |
| Determining Alternative Outcomes | Analyzing historical events or decisions and reasoning about possible alternative outcomes if key factors had been different. |
| Distinguishing Correlation from Causation | Analyzing given scenarios or statistical relationships to identify cases where correlation does not imply causation. |
| Distinguishing Fact From Opinion | Differentiating between factual statements and opinions in given texts or scenarios. |
| Equation Derivation | Deriving mathematical equations from given information or scenarios to evaluate the ability to translate word problems into mathematical expressions and manipulate them to reach a solution. |
| Estimating Duration | Estimating the time required for various activities or processes. |
| Ethical Dilemma Resolution | Analyzing complex ethical scenarios and proposing reasoned solutions. |
| Evaluating Analogies For Accuracy | Assessing given analogies for their accuracy and appropriateness. |
| Evaluating Competing Theories | Analyzing and comparing multiple theories for a phenomenon, assessing their strengths and weaknesses, and determining which is best supported by evidence. |
| Evaluating Policy Implications | Analyzing proposed policies or decisions and predicting their potential consequences across various domains. |
| Evaluating Source Credibility | Assessing the reliability and trustworthiness of various information sources. |
| Fermi Estimation | Making rough numerical estimates for complex questions using logical reasoning and educated guesses. |
| First Order False Belief | Identifying why a misinformed agent may behave contrary to reality due to inaccurate beliefs. |
| First Order Ignorance | Identifying why an agent may lack knowledge or awareness of certain facts or events. |
| Forecasting Technological Impacts | Predicting and analyzing potential effects of emerging or hypothetical technologies on society, economy, and daily life. |
| Generating Creative Solutions | Developing innovative and unique solutions to given problems or challenges. |
| Higher Order False Belief | Handling a complex chain of agent's beliefs about the knowledge (and accuracy of the knowledge) of other agents to ultimately predict the behavior of an agent. |
| Hypothesis Formation | Generating plausible hypotheses to explain observed phenomena or solve problems. |
| Identifying Anachronisms | Recognizing elements that are out of place in a given historical context. |
| Identifying Cause and Effect Relationships | Analyzing given scenarios or statements to identify and explain cause and effect relationships present. |
| Identifying Cognitive Biases | Recognizing and explaining cognitive biases in given scenarios or decision-making processes. |
| Identifying Hallucination Prone Questions | Recognizing questions that are likely to lead to hallucination or questions it simply doesn't know the answer to. |
| Identifying Implicit Biases In Language | Recognizing subtle biases embedded in language use. |
| Identifying Logical Fallacies | Identifying and explaining common logical fallacies in given arguments or statements to evaluate the ability to recognize flawed reasoning. |
| Identifying Logical Inconsistencies | Detecting and explaining logical inconsistencies or contradictions within given information. |
| Identifying Relationships | Identifying the relationship between pairs of words or concepts to evaluate the ability to recognize various types of connections and articulate them clearly. |
| Identifying Unstated Assumptions | Recognizing and articulating implicit assumptions in statements, arguments, or scenarios. |
| Inference Drawing From Incomplete Data | Making logical deductions or inferences based on limited or incomplete information. |
| Inferring Emotional States | Analyzing scenarios to infer people's emotional states based on their behavior and context. |
| Inferring Motivations From Actions | Analyzing described actions or behaviors to deduce the underlying motivations, intentions, or goals of the individuals involved. |
| Interpreting Ambiguous Statements | Analyzing statements with multiple possible interpretations and identifying different ways they can be understood. |
| Interpreting and Creating Timelines | Reading or creating timelines to represent a series of events or processes. |
| Interpreting Body Language Cues | Analyzing and interpreting non-verbal communication signals, such as facial expressions, postures, and gestures. |
| Interpreting Legal Language And Precedents | Analyzing and interpreting legal texts, statutes, or case law to understand their implications and applications. |
| Interpreting Nonverbal Communication | Analyzing and interpreting nonverbal cues in human communication. |
| Interpreting Statistical Data | Analyzing and interpreting statistical data presented in various formats. |
| Lateral Thinking Puzzles | Solving lateral thinking puzzles to evaluate the ability to think creatively, consider unconventional scenarios, and ask relevant questions to reach the correct solution. |
| Mathematical Word Problems | Solving mathematical word problems to evaluate the ability to interpret real-world scenarios, extract relevant information, and perform appropriate calculations. |
| Mental Rotation Tasks | Mentally rotating objects or shapes and predicting their appearance from different angles. |
| Moral Reasoning In Everyday Situations | Analyzing everyday scenarios that present moral dilemmas and reasoning through the ethical implications of different actions. |
| Narrative Gap Filling | Filling in missing information or events in a narrative to create a coherent story. |
| Parsing Complex Sentences | Analyzing sentences to identify grammatical structures and explain relationships between parts. |
| Pattern Recognition in Spatial Arrangements | Identifying patterns or rules inentifying patterns or rules in the spatial arrangement of objects or shapes. |
| Perspective Taking In Social Scenarios | Analyzing social situations from different viewpoints to understand the motivations, emotions, and potential reactions of various parties involved. |
| Predicting Market Trends | Analyzing economic indicators and data to forecast potential future market trends. |
| Predicting Outcomes Based on Scenarios | Analyzing given scenarios and predicting potential outcomes based on the information provided. |
| Prioritizing Conflicting Goals | Analyzing and resolving situations with conflicting objectives, requiring trade-offs and strategic decision-making. |
| Proof Verification | Verifying mathematical proofs to evaluate the ability to understand logical arguments, identify correct steps in a proof, and spot errors or gaps in reasoning. |
| Recognizing Emotional Subtext | Identifying and interpreting underlying emotional content or implications in text that are not explicitly stated. |
| Recognizing Patterns In Behavior | Identifying recurring patterns or trends in human or animal behavior based on given scenarios or data. |
| Recognizing Patterns In Sequences | Identifying and extending patterns in numerical, alphabetical, or symbolic sequences. |
| Reconciling Conflicting Information | Analyzing and resolving contradictory information from multiple sources. |
| Reverse Engineering Processes | Analyzing a final result and working backwards to determine the steps or components that led to that outcome. |
| Risk Assessment In Decision Making | Evaluating potential risks and benefits associated with different courses of action in a given scenario. |
| Second Order False Belief | Understanding that an agent may hold a false belief about another agent's belief, leading to misinterpretations of actions or intentions. |
| Sequencing Events | Arranging a set of events in chronological order. |
| Solving Riddles And Word Puzzles | Deciphering and solving various types of riddles and word puzzles. |
| Solving Word Problems With Multiple Variables | Translating word problems into mathematical equations and solving them. |
| Spatial Problem Solving | Using spatial reasoning to solve practical problems or puzzles. |
| [Stack Based Reasoning] (https://github.com/NousResearch/Open-Reasoning-Tasks/blob/main/tasks/stack-based-reasoning) Using stacks and reverse thinking to convert chain of thought into stacks of instructions for others to follow | |
| Syllogism Reasoning | Providing a series of syllogisms to evaluate logical reasoning capabilities. |
| Trait Attribution In Behavioral Scenarios | Analyzing described behaviors or actions and inferring personality traits or characteristics that might explain those behaviors. |
| Trend Analysis And Forecasting | Examining historical data or patterns to identify trends and make predictions about future outcomes. |
| Truth Table Completion | Completing truth tables for given logical expressions to evaluate the understanding of Boolean logic and ability to determine the truth value of complex statements. |
| Truth Table Completions | Completing truth tables for given logical expressions. |
| Towers of Hanoi | The Tower of Hanoi is a puzzle where you must move a stack of disks from one rod to another, obeying rules about disk size and movement. |
| Understanding Time Based Relationships | Analyzing and explaining relationships between events based on their timing. |
| Understanding Time Zones and Global Time Differences | Calculating time differences across various time zones and understanding how global time works. |
| Unraveling Paradoxes | Analyzing and explaining paradoxes or logical puzzles. |
| Multiturn Latex Generation | Generating latex documents and suggesting edits using multiturn instructions. |