1 00:00:00,000 --> 00:00:10,000 In this video, you'll learn about systemd journald. 2 00:00:10,000 --> 00:00:12,480 So what is systemd journald? 3 00:00:12,480 --> 00:00:16,080 Well, the systemd journal handles log messages 4 00:00:16,080 --> 00:00:19,040 that are generated by any systemd unit. 5 00:00:19,040 --> 00:00:20,360 It kind of makes sense. 6 00:00:20,360 --> 00:00:22,600 Systemd is taking care of everything. 7 00:00:22,600 --> 00:00:24,719 And as it is taking care of everything, 8 00:00:24,719 --> 00:00:27,080 it also sees all messages generated 9 00:00:27,080 --> 00:00:29,040 by everything on your system. 10 00:00:29,040 --> 00:00:32,000 And that is why systemd journald was introduced 11 00:00:32,000 --> 00:00:34,240 as a generic log service. 12 00:00:34,240 --> 00:00:36,439 Now, the first level of getting information 13 00:00:36,439 --> 00:00:39,200 about your services in systemd is 14 00:00:39,200 --> 00:00:42,880 by just typing systemctl status on your unit. 15 00:00:42,880 --> 00:00:47,439 Use systemctl status sshd.service, for instance, 16 00:00:47,439 --> 00:00:50,279 to see the most recent messages that 17 00:00:50,279 --> 00:00:52,919 have been logged by that service. 18 00:00:52,919 --> 00:00:55,639 You can also use journalctl. 19 00:00:55,639 --> 00:00:58,200 Journalctl prints a complete journal. 20 00:00:58,200 --> 00:01:01,320 And it has some nice filtering options. 21 00:01:01,320 --> 00:01:04,160 These include minus F, which will follow the journal 22 00:01:04,160 --> 00:01:08,279 while new messages are added, or minus PR, 23 00:01:08,279 --> 00:01:12,680 where you will only print events with a priority of error 24 00:01:12,680 --> 00:01:14,400 and higher. 25 00:01:14,400 --> 00:01:17,680 Also pretty powerful is using minus 26 00:01:17,680 --> 00:01:21,559 since followed by a time indicator until followed 27 00:01:21,559 --> 00:01:25,120 by a time indicator as well, so that you can find messages 28 00:01:25,120 --> 00:01:27,480 from a specific time frame. 29 00:01:27,480 --> 00:01:30,680 If it's not enough, use journalctl minus over both 30 00:01:30,680 --> 00:01:34,879 for more details, or minus U, which prints events 31 00:01:34,879 --> 00:01:37,040 for a specific unit. 32 00:01:37,040 --> 00:01:39,559 If you want more information about all of them, 33 00:01:39,559 --> 00:01:42,199 systemd has a nice man page that you can find 34 00:01:42,199 --> 00:01:45,400 in systemd.journal-fields. 35 00:01:45,400 --> 00:01:47,519 I'm going to demonstrate a couple of them, 36 00:01:47,519 --> 00:01:51,199 my personal favorites. 37 00:01:51,199 --> 00:01:53,279 So here we have journalctl. 38 00:01:53,320 --> 00:01:57,559 Journalctl is showing its output in a less-like pager, 39 00:01:57,559 --> 00:02:01,919 where you can use your spacebar to scroll down or uppercase G 40 00:02:01,919 --> 00:02:04,239 to go all the way down. 41 00:02:04,239 --> 00:02:07,239 Now, what I really like is my filtering options, 42 00:02:07,239 --> 00:02:14,800 like journalctl minus U sshd, which is showing it 43 00:02:14,800 --> 00:02:16,639 for the sshd unit. 44 00:02:16,639 --> 00:02:17,960 Kind of boring. 45 00:02:18,000 --> 00:02:23,960 You can also add the x, so minus x U sshd, where x 46 00:02:23,960 --> 00:02:26,160 is giving some explanation. 47 00:02:26,160 --> 00:02:28,919 Now, don't expect too much of this explanation, 48 00:02:28,919 --> 00:02:32,360 but it might just be what you need in order to figure out 49 00:02:32,360 --> 00:02:34,520 what is really going on. 50 00:02:34,520 --> 00:02:36,360 Now, related to that, you can also 51 00:02:36,360 --> 00:02:39,479 use journalctl minus xb. 52 00:02:39,479 --> 00:02:44,039 Minus xb is showing the boot log, all the messages that 53 00:02:44,039 --> 00:02:46,839 are logged since booting. 54 00:02:46,839 --> 00:02:51,679 Or what do you think of journalctl minus minus dmessage? 55 00:02:51,679 --> 00:02:54,839 Dmessage is reading the kernel log. 56 00:02:54,839 --> 00:02:58,360 The kernel logs are written to a specific area of memory. 57 00:02:58,360 --> 00:03:01,679 And before, you would use a separate command, dmessage, 58 00:03:01,679 --> 00:03:04,080 which, by the way, still works. 59 00:03:04,080 --> 00:03:06,160 And as you can see, it's even nicer. 60 00:03:06,160 --> 00:03:07,919 It's showing nice colors. 61 00:03:07,919 --> 00:03:10,919 And it's showing a time offset in the beginning. 62 00:03:10,919 --> 00:03:12,520 If you're wondering what is this, 63 00:03:12,559 --> 00:03:14,960 well, this is the number of seconds 64 00:03:14,960 --> 00:03:20,279 since the system booted that the message has been generated. 65 00:03:20,279 --> 00:03:22,440 There's something else we need to discuss regarding 66 00:03:22,440 --> 00:03:28,240 journalctl, because it's not persistent by default. 67 00:03:28,240 --> 00:03:30,399 The disadvantage of the system, the journal, 68 00:03:30,399 --> 00:03:32,919 is that it is not persistent by default. 69 00:03:32,919 --> 00:03:35,839 So you reboot, you lose it. 70 00:03:35,839 --> 00:03:38,119 But you can make it persistent if you want to. 71 00:03:38,119 --> 00:03:40,039 And it's actually pretty simple. 72 00:03:40,039 --> 00:03:43,679 When in the journal.conf, the option storage is always set, 73 00:03:43,679 --> 00:03:45,679 a persistent journal will be created 74 00:03:45,679 --> 00:03:50,160 after you create a directory with the name var log journal. 75 00:03:50,160 --> 00:03:54,440 But you will have to use a systemctl restart system 76 00:03:54,440 --> 00:03:57,279 journal flush command to trigger systemd 77 00:03:57,279 --> 00:03:59,399 to reload its configuration. 78 00:03:59,399 --> 00:04:02,559 Up to that moment, your journal won't be persistent. 79 00:04:02,559 --> 00:04:03,839 Let me show you. 80 00:04:03,839 --> 00:04:09,639 So let's start by using find slash minus name journald.conf. 81 00:04:09,639 --> 00:04:12,960 And that is because the name of the configuration file 82 00:04:12,960 --> 00:04:15,399 might be different on your distribution. 83 00:04:15,399 --> 00:04:24,119 So here we have user lib systemd journald.conf. 84 00:04:24,119 --> 00:04:25,519 With all of the different options 85 00:04:25,519 --> 00:04:29,519 that apply to the journal, including storage is auto. 86 00:04:29,519 --> 00:04:31,480 No need to change anything. 87 00:04:31,480 --> 00:04:34,200 Let's just make this journal persistent. 88 00:04:34,239 --> 00:04:40,600 So I'm going to use mkdir var log journal. 89 00:04:40,600 --> 00:04:42,679 And that creates a directory journal. 90 00:04:42,679 --> 00:04:50,920 But ls on var log journal is not showing anything by default. 91 00:04:50,920 --> 00:04:53,799 And that is because we need to trigger it. 92 00:04:53,799 --> 00:04:56,480 Now systemd nowadays is quite strong 93 00:04:56,480 --> 00:04:58,920 at picking up modifications without having 94 00:04:58,920 --> 00:05:00,640 to reboot anything. 95 00:05:00,640 --> 00:05:03,679 And it's kind of weird that this modification 96 00:05:03,679 --> 00:05:05,320 needs a specific command. 97 00:05:05,320 --> 00:05:12,880 And that is systemctl restart systemd-journal-flush. 98 00:05:12,880 --> 00:05:15,959 And now if I use my ls in var log journal again, 99 00:05:15,959 --> 00:05:17,559 there you can see a directory. 100 00:05:17,559 --> 00:05:20,279 And that is the directory where my persistent journal 101 00:05:20,279 --> 00:05:21,480 is going to be. 102 00:05:21,480 --> 00:05:24,880 And that means that from now on, everything in the journal 103 00:05:24,880 --> 00:05:27,279 is consistently written to disk. 104 00:05:27,279 --> 00:05:30,160 It will follow the restrictions that 105 00:05:30,160 --> 00:05:33,079 are in the journald.conf file. 106 00:05:33,079 --> 00:05:36,519 So if you are worried that your system is going to be filled up 107 00:05:36,519 --> 00:05:40,440 or that messages are going to be cut off too easy, 108 00:05:40,440 --> 00:05:43,920 have a look at the journald.conf file.