Speaker #1 - Kong on Zhiji

https://www.linkedin.com/in/kongleesg/

www.zhiji.io

ZHIJI_TECH_STACK_ROAST.html

1. Latency Optimization

Problem: Slow response times negatively impact user experience.

Possible solutions:

• Pre compute common or predictable results instead of generating everything on demand.
• Use token streaming so users see the response being generated immediately instead of waiting for the full completion.
• Evaluate Iris.io as a potential solution for reducing latency.

2. Memory Management and Context Pruning

Challenge: Long conversations lead to context overload and context rot.

Approach:

• Use Haiku to periodically compress or summarize conversation memory.
• Compact memory before adding it back into the context window.
• Prune unnecessary context to prevent degradation in model performance.

3. Trusting AI Generated Architecture

Observations:

• Kong uses an agent orchestrator to manage product development workflows.