1 00:00:00,000 --> 00:00:11,040 So sometimes, SQLinux will cause problems, it doesn't work, and then you need to know 2 00:00:11,040 --> 00:00:13,160 how to troubleshoot. 3 00:00:13,160 --> 00:00:15,120 So how does troubleshooting work? 4 00:00:15,120 --> 00:00:19,400 Well, first you are going to check events that are in the audit log. 5 00:00:19,400 --> 00:00:20,520 We already did that. 6 00:00:20,520 --> 00:00:28,559 All the SQLinux events are written with AVC, so you use grep AVC on varlogaudit audit.log 7 00:00:28,559 --> 00:00:32,720 to make sure that you understand what is going on. 8 00:00:32,720 --> 00:00:35,400 And messages in the audit log are not always very readable. 9 00:00:35,400 --> 00:00:38,400 Well, you know, I don't even agree to that. 10 00:00:38,400 --> 00:00:44,439 Messages in the audit log are pretty readable, but you need to know what you are checking. 11 00:00:44,439 --> 00:00:46,080 Let's check it out. 12 00:00:46,080 --> 00:00:56,560 So grep AVC on varlogaudit audit.log is showing what? 13 00:00:56,560 --> 00:00:59,840 Well, let's take this one. 14 00:00:59,840 --> 00:01:01,880 That's one that we haven't talked about yet. 15 00:01:01,880 --> 00:01:04,879 This is telling me denied BPF. 16 00:01:04,879 --> 00:01:07,839 BPF, that's a system call. 17 00:01:07,839 --> 00:01:11,239 The PID is the process that was involved. 18 00:01:11,239 --> 00:01:14,580 And this process apparently is to command mdadm. 19 00:01:14,580 --> 00:01:22,080 And this mdm is using a source context mdadm underscore T. And the target context is what 20 00:01:22,080 --> 00:01:26,160 you see right here, is mdadm underscore T. 21 00:01:26,160 --> 00:01:27,360 So what does that mean? 22 00:01:27,360 --> 00:01:30,720 Well, that is, in fact, a very curious error message. 23 00:01:30,720 --> 00:01:39,320 That means that mdadm is not allowed to issue the BPF system call, even if it has a source 24 00:01:39,320 --> 00:01:45,720 context mdadm T going to the target context mdadm underscore T. 25 00:01:45,720 --> 00:01:48,559 Source and target context are the same normally that should work. 26 00:01:48,559 --> 00:01:50,720 So here, well, you know what is wrong. 27 00:01:50,720 --> 00:01:54,519 And next, you need to find out what to do about it. 28 00:01:54,519 --> 00:01:56,080 Let's take another one. 29 00:01:56,080 --> 00:01:57,080 Like this one. 30 00:01:57,080 --> 00:01:59,040 This was the Apache web server. 31 00:01:59,040 --> 00:02:01,040 And what do we see in the Apache web server? 32 00:02:01,040 --> 00:02:04,199 Well, we see ABC denied read. 33 00:02:04,199 --> 00:02:05,720 So it has tried to read something. 34 00:02:05,720 --> 00:02:10,600 Well, what did it read for PID 1253 command HTTPD? 35 00:02:10,600 --> 00:02:11,880 Name index HTML. 36 00:02:11,880 --> 00:02:12,880 Aha. 37 00:02:12,880 --> 00:02:16,440 HTTPD was accessing index HTML. 38 00:02:16,440 --> 00:02:22,080 There we have the source context HTTPD underscore T, the target context default underscore T. 39 00:02:22,080 --> 00:02:25,440 Well, you know what to do in that case, don't you? 40 00:02:25,440 --> 00:02:29,960 And that is why I think that messages in the audit log are pretty readable and should 41 00:02:29,960 --> 00:02:33,080 always be the start of your troubleshooting. 42 00:02:33,080 --> 00:02:38,720 The next utility, se-alert, is only useful if you know what is going on. 43 00:02:38,720 --> 00:02:42,119 Edit or start a slide from here. 44 00:02:42,119 --> 00:02:43,119 So what is se-alert? 45 00:02:43,119 --> 00:02:49,399 Se-alert is part of the se-troubleshoot server package, and it logs readable messages to 46 00:02:49,399 --> 00:02:51,360 your logging system. 47 00:02:51,360 --> 00:02:57,479 And you can look for se-alert in your log files and learn the suggested command to find 48 00:02:57,479 --> 00:02:59,399 out what to do. 49 00:02:59,399 --> 00:03:02,360 In some cases, you get very useful information. 50 00:03:02,360 --> 00:03:05,479 In some cases, it will be completely useless. 51 00:03:05,479 --> 00:03:08,559 Let's check out what it is doing in this case. 52 00:03:08,559 --> 00:03:15,679 So I'm going to use journal CTL by grab se-alert. 53 00:03:15,679 --> 00:03:21,160 And there we can see that se-alert has generated a couple of messages. 54 00:03:21,960 --> 00:03:23,880 And what you need to do is to run this command. 55 00:03:23,880 --> 00:03:27,320 So I'm going to do it on the webindex.html case first. 56 00:03:27,320 --> 00:03:31,759 And then just because I'm curious, I also want to do it for mdadm. 57 00:03:31,759 --> 00:03:35,919 Now I know that a lot of output is coming, so I'm filtering through less. 58 00:03:35,919 --> 00:03:36,919 And what is it doing? 59 00:03:36,919 --> 00:03:46,399 It's telling me, plug-in catch-all boolean with 57.6% confidence suggests that this is 60 00:03:46,399 --> 00:03:47,399 the solution. 61 00:03:47,399 --> 00:03:50,440 Well, I can tell you it's not. 62 00:03:50,440 --> 00:03:52,880 Then it comes with another suggestion. 63 00:03:52,880 --> 00:03:58,479 And there it's telling that se-manage-fcontext-a-t-file-type is the solution. 64 00:03:58,479 --> 00:04:00,600 So we need to set a file type. 65 00:04:00,600 --> 00:04:06,220 But then the problematic part, make your choice of the long list of file types that follows. 66 00:04:06,220 --> 00:04:08,240 That is not very useful either. 67 00:04:08,240 --> 00:04:14,720 But believe me, sometimes se-alert is just giving you a command that does make sense. 68 00:04:14,720 --> 00:04:16,519 So you can give it a try. 69 00:04:16,519 --> 00:04:21,279 But my advice is to always match se-alert to your own understanding of what is going 70 00:04:21,279 --> 00:04:22,279 wrong. 71 00:04:22,279 --> 00:04:27,760 Now, as a bonus, and because I haven't seen it before, I want to investigate this mdadm 72 00:04:27,760 --> 00:04:28,760 as well. 73 00:04:28,760 --> 00:04:34,799 So apparently, this is something that was not configured all right in CentOS 10. 74 00:04:34,799 --> 00:04:38,559 And what is se-alert telling me about it? 75 00:04:38,559 --> 00:04:42,839 Well, se-alert is telling me that I need to report this as a bug. 76 00:04:42,839 --> 00:04:43,839 I agree. 77 00:04:43,839 --> 00:04:44,839 It sounds like a bug. 78 00:04:45,160 --> 00:04:49,200 But it also is telling me that this is the solution to fix it. 79 00:04:49,200 --> 00:04:51,959 It's going too far for what we are doing right here. 80 00:04:51,959 --> 00:04:56,540 With this solution, we'll generate an se-Linux policy module. 81 00:04:56,540 --> 00:05:03,059 And in this se-Linux policy module, everything that was denied for mdadm is going to be allowed. 82 00:05:03,059 --> 00:05:08,920 And that will make your message go away after you have used se-module minus I to insert 83 00:05:08,920 --> 00:05:11,059 this new policy module. 84 00:05:11,059 --> 00:05:13,200 So what to remember about this? 85 00:05:13,600 --> 00:05:20,320 Start with your own understanding on the varlogaudit audit.log and use se-alert as a help to try 86 00:05:20,320 --> 00:05:22,320 to narrow down the problem. 87 00:05:22,320 --> 00:05:23,600 Sometimes it's useful. 88 00:05:23,600 --> 00:05:24,880 Sometimes it completely is not.