Setup RKLLM Runtime on CM3588 (Rockchip RK3588)
Why did I do this? Like always being bored is the reason for anything I do with tech. With the AI hype threatening my job (LOL), I wanted to play around with LLMs for actual daily use and not just generating slop. Since I own FriendlyElec’s CM3588 board with RK3588 with 32GB ram and 256GB storage it was overkill for my home lab use with CF tunnel, K3s cluster and containers just serving me my flac files, Immich and occasional side projects which I never use. I was aware that this board has an NPU but I never had the time to play with it and run a model off it. I did try ollama in my desktop but it never made any sense because of the resource usage and power consumption. ...