Lack of Linux understanding

[ThermoDust 12]

Well, today my machine took a hard crap and only booted with and error. /dev/nvme: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. While I know how to get around Linux my troubleshooting skills are lacking big time.

 

I ran the command fsck and it booted after it cleaned up MANY errors. However, it seemed to of broken Emby also. I had to force a full removal as it complained about SQLLite. Once I did a full purge and directory cleanups I get this error now.

● emby-server.service - Emby Media Server
   Loaded: loaded (/lib/systemd/system/emby-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2018-05-03 22:09:56 CDT; 9s ago
 Main PID: 2836 (emby-server)
   CGroup: /system.slice/emby-server.service
           ├─2836 /bin/bash /usr/bin/emby-server start
           ├─2884 /bin/bash /usr/bin/emby-server start
           ├─2885 find -type d -name coders
           └─2886 grep EmbyMagick

May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2641/task/2641/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2641/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2642/task/2642/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2642/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2643/task/2643/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2643/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2644/task/2644/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2644/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2645/task/2645/net’: Invalid argument
May 03 22:10:01 Pixy-PC emby-server[2836]: find: ‘./proc/2645/net’: Invalid argument

Ideas?

[woenk 62]

Either the .NET part of emby is misconfigured or some networking stuff. The install went fine without errors ? Everything else is working ?

Could as well be that the purge did not delete everything and somewhere there is still some library or conf file lingering about which was not restored to the orginal value.

 

What I find more frighting is, that your NVME SSD took a dive. Maybe do some SMARt checks to find out, if it is all right, could be that you did a lot of writes and it does have any spare cells left or is writing to defect cells.

Unless you did something "really bad and stupid", it could be that your SSD is about to go to silicon heaven.

Ok, could be a lot of things, but filesystem errors should not happen no matter what.

 

Since you are troubleshooting, ever thought of a backup ?

Take a look at Veeam Agent for Linux, they have a free version (only one backup per day), it is easily set up and restoring a backup is done really fast.

Restoring a 20GB VM takes me less than 5 minutes from an NFS share.

Even made a TFP boot server and now i can restore any computer via PXE in a zip.

[ThermoDust 12]

Since you are troubleshooting, ever thought of a backup ?

Take a look at Veeam Agent for Linux, they have a free version (only one backup per day), it is easily set up and restoring a backup is done really fast.

Restoring a 20GB VM takes me less than 5 minutes from an NFS share.

Even made a TFP boot server and now i can restore any computer via PXE in a zip.

 

Funny you say backup as I just nuked my backup drive that morning as I needed to do a backup of another machine... talk about bad luck.

 

Smart: 

root@Pixy-PC:~# sudo smartctl -a /dev/nvme0
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-6-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Force MP500
Serial Number:                      17127956000123380135
Firmware Version:                   E7FM02.1
PCI Vendor/Subsystem ID:            0x1987
IEEE OUI Identifier:                0x6479a7
Controller ID:                      0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          120,034,123,776 [120 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Fri May  4 09:14:49 2018 CDT
Firmware Updates (0x02):            1 Slot
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x001e):     Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     110 Celsius
Critical Comp. Temp. Threshold:     130 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     7.90W       -        -    0  0  0  0        0       0
 1 +     2.40W       -        -    1  1  1  1      600     600
 2 +     1.90W       -        -    2  2  2  2      600     600
 3 -   0.1100W       -        -    3  3  3  3      600     600
 4 -   0.0050W       -        -    4  4  4  4   100000  160000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:                   0x00
Temperature:                        45 Celsius
Available Spare:                    100%
Available Spare Threshold:          0%
Percentage Used:                    2%
Data Units Read:                    3,044,091 [1.55 TB]
Data Units Written:                 6,453,922 [3.30 TB]
Host Read Commands:                 77,758,445
Host Write Commands:                174,001,563
Controller Busy Time:               0
Power Cycles:                       34
Power On Hours:                     7,214
Unsafe Shutdowns:                   10
Media and Data Integrity Errors:    2,874
Error Information Log Entries:      2,874
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 2:               59 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc          LBA  NSID    VS
  0       2874     1  0x029b  0x4502      -   4294967288     1     -
  1       2873     1  0x029b  0x4502      -   4294967288     1     -
  2       2872     1  0x029b  0x4502      -   4294967288     1     -
  3       2871     1  0x029b  0x4502      -   4294967288     1     -
  4       2870     1  0x029b  0x4502      -   4294967288     1     -
  5       2869     1  0x029b  0x4502      -   4294967288     1     -
  6       2868     1  0x029b  0x4502      -   4294967288     1     -
  7       2867     1  0x029b  0x4502      -   4294967288     1     -
  8       2866     1  0x029b  0x4502      -   4294967288     1     -
  9       2865     1  0x029b  0x4502      -   4294967288     1     -
 10       2864     1  0x029b  0x4502      -   4294967288     1     -
 11       2863     1  0x029b  0x4502      -   4294967288     1     -
 12       2862     1  0x029b  0x4502      -   4294967288     1     -
 13       2861     1  0x029b  0x4502      -   4294967288     1     -
 14       2860     1  0x029b  0x4502      -   4294967288     1     -
 15       2859     1  0x029b  0x4502      -   4294967288     1     -
... (47 entries not shown)