1
00:00:00,000 --> 00:00:11,320
On this video, you'll learn about requirements for hosting GenAI on Linux.

2
00:00:11,320 --> 00:00:12,560
So what do you need?

3
00:00:12,560 --> 00:00:17,840
Well, to host a GenAI solution on Linux, you'll need a framework that includes at least a

4
00:00:17,840 --> 00:00:20,320
client and an LLM.

5
00:00:20,320 --> 00:00:22,600
Which LLM to use is flexible.

6
00:00:22,600 --> 00:00:27,280
Open source LLMs are provided through HuggingFace.co and you might want to check your search for

7
00:00:27,280 --> 00:00:30,520
a reliable LLM just there.

8
00:00:30,520 --> 00:00:34,639
The framework should allow users to integrate data using LLM fine-tuning.

9
00:00:34,639 --> 00:00:41,080
LLM fine-tuning is really important for most sites that run their own AI.

10
00:00:41,080 --> 00:00:46,639
And typical tasks performed on AI framework are inference and fine-tuning, where inference

11
00:00:46,639 --> 00:00:52,400
is about interacting with the AI system to get accurate information, and fine-tuning

12
00:00:52,400 --> 00:00:55,240
is about adding your own data.

13
00:00:55,240 --> 00:01:00,639
And don't forget, AI solutions require intense resources.

14
00:01:00,639 --> 00:01:05,639
And to perform either of these tasks in a decent way, GPU support is needed.

15
00:01:05,639 --> 00:01:07,160
Lots of GPU.

16
00:01:07,160 --> 00:01:11,440
For the simple reason that GPU RAM is a lot faster than ordinary RAM.

17
00:01:11,440 --> 00:01:13,279
And really, you're going to need that speed.