Full-Stack Artificial Intelligence

We cannot expect the same algorithm to make increasing returns on all problems. The complexity (and ⏱️,💲,🏭) growth of symbolic, deep learning, reinforcement learning, and other AI approaches motivate considering other dimensions of Intelligence. I draw attention to several broad, overlapping dimensions on the evolutionary scale of artificial intelligence:

• Intrinsic properties: What are the intrinsic properties of a problem? How should the information flow from problem to solution? How should information flow from inputs to outputs for a single instance of the problem? What priors can we assume?
• Algorithm design: How do we concretely express the ideal information flow? What computational tricks will help us? How do we choose the right hyper/parameters? How are learnable components (hidden states, parameters, architectures) initialized?
• Datasets and environments: What data will we need? Does it need to be preprocessed?
• Training paradigms: How will we impose feedback on the solution space? SL, US, SSL, RL, MARL, multi-paradigm? Should the loss be a weighted sum of losses or should it look more like a pareto curve?
• Evaluation: How do we evaluate the performance of the solution? What quantitative metrics will we use? What qualitative assessments can we reasonably make?
• Infrastructure: What are the hardware and software requirements?
• Human feedback: How will we understand and communicate the system's performance? What should my iteration speed as the developer be?
• Existing code: Do I know what's already been done? What should I do myself?
• Human capital: What intelligence have others (developers, researchers, other thinkers) already contributed? Could this be a group effort? What motivational forces (free time, interest, project complexity and understandability) should be considered?

I try to trackle these sprawling points in future posts.