I am seeing an issue in my logs pertaining to an aspected post that I created recently, whereby at least one user on another pod (Friendica, in this instance) has clicked the “like” button, but the “like” is not federated to my pod. In the log, it manifests as a WARN message, and there are two varieties that I am seeing:
At the earliest time period where the error started, I see this message repeatedly:
[2018-11-01T03:38:00] WARN PID-3148 TID-70036387642100 DiasporaFederation::Federation::Sender::HydraWrapper: success=false sender=2 obj=Like:19512760bfb301363e7b2a0000053625:Post:59f0edb0bfa501368c061697a07b37d3 url=https://squeet.me/receive/users/xxxxxxxxxxxxxx message=ok code=500 time=0.746068
Then, later, I see this Sidekiq exception:
[2018-11-01T05:31:50] WARN PID-3148 TID-70036593127180 Sidekiq: {"context":"Job raised exception","job":{"class":"Workers::SendPrivate","args":[2,"StatusMessage:59f0edb0bfa501368c061697a07b37d3",{"https://squeet.me/receive/users/xxxxxxxxxxxxxx":"{\"aes_key\":\"KI6Vc7ygfkIFl+vvSiOQorGUbqkePzurOXeXWElgF+4x+GS7dQq/vLCdeNvuCogDNGRVnLb6gyXu5NzfhdEOIOnKa4S+/8IvdQGScyPPkG54d1CdOUTaiwqp1LpWON7EC5AFzK6uUso4Y+nhJugY52wfA/l0qr0YE4NNeE0/SoijVj2nSyvQqBfLQ4htvOqCxthiMZ5Hp/89UzB/2PqnsxFmHhTDKSrcCpzD6fYiSnHzKRhSN8RwRvfB2baDNd8/rmjCc/afLMnNtJBsOu+BqpVoeo4Dt6JyIQNxVHe0Cd/nPt3fOKFm0IQPJ2CRAQiYnMxAyLzHTipoc5nNp7mp7YhMkqxVoo6qIR1rfHfvSffxPazAq6ZyZqpxZfD3bE1xxOLHmvQjh3QrtcPqb5u+3rXTkY9+zm54oc5Kq3AsQhokkXC5PWWNkj8yW4y1NwW+SGZbx4HrRnM3U4dYir+KwB2/f19rWrjY2hiSO7sSHhYgEnfMLMwd0zEaxi6mu9D+nX9YBi334+tMgB63dhUndiTg68D59weZGtzl7lN8pbLComIUBJO7CvgyRKwlm3P2YdvtQycOEyylPDtJt4+XWmRHcQDVqif1qyWm4E8lGigfUcoSjP1vnMcrs7vznXe6dVMcMxHm/3ZmubeH9eAc/7Ak+WJK5SpmO239l3NpLqI=\",\"encrypted_magic_envelope\":\"JER30XbKPaIUl60E5HgodvPGIPvjm+yI9HaF2PPcz/Bcql78zlZiSgDDXjMaBFUebSPlj7vGt9IJOaBTvyDVBewv5PiTj7kzCPKXGBtGnyHjzBcyOGQ3uNjL7WViAfeMHg1hdZiisdaaHOA2R29GsnSvL2dEZCdC54zWTdvuXOpb0ZgCXWqoZSjpXhjhQb7hmr74xxVtnhAh4Zt/583y9wv1OuvSa/wjJXlHr/fa0PE1TpJBROZhF8gTPsBxIh+FL4j9dakOjpDAdyQzXgz3KTANgmYo7CAgzNV8Js6yoH6wDFS3yRrUjFW96imZie4BZhoR9fbul6+/IgqjjoEPt8OJDOPqOkAgY+It1n1Y/Kgz2AeEzBWfwozXXQ5qsv5QKvLjfwDa8/LfIrsuORgjdSXjYCQqfimsVYJoSIZ340Ga2NsmyaBrAbHBbQKXyNQPEkK8kAErahIx/wzKlzSmdkgr4DyI7fszIbdp0Gh2vqLJnYQi7p9XOpwlwpikdZzByCfMukJMzWtWX2DTvcLcMYTIglekTF94wnG4gME7ZL1/Ue3J0jBCPS1cQ6oMBgHhHZVl1lzV/9jpg2e2i4MWiOzYnq6ARbWGr/1uyZollDEAglF89YPhjTICjE0wKZswzRW/ONM2DCgilV39waxXxCR31n/8YSzIYhDl+DDr/zsyUkQZYRMqusDKMlyjtSPSeXt8i7G3RtusyMfBIlfOyU3iNvlhSlSwc+g1qhOIQKRM82QaCiHB1zz/fSWcEzNSzC5KK7xYLIUGKcnNSJaMkLKmHxnqhZXVxge1Ol9OEYxkLCr8g2ELO4dS4LsokThN5q0eZDbuoX58/Z0jvH4OdZMLG4S3+FaKHxZtvk/NFQRGR2lWO2O9bXIjcQxxjRn+NJW1r3o0e6A3locjY4WOpV2q/pTHsMBZDqkofd6LAF36KK71Bm3BV6/2L4tJkJo1XO7hM6Gj6allH4tB0uIgIwIJhUBe2DzZPrquWswzais/2DYShiKkqQdL7AP8vfuH7e1zDvlQpg3pJ4f3jFWevHE5QAQk/4ZPvtlLyrGIw4WkTCGre2r0qVkf0HOoJgNDp/+e9EeJnHuDjcZi+f6OYlylCpunHHUsjDFff4RIfOqZUIkcZP7Wfx/71lP99TodtH1q1SIABAkWHcFEsaGgFsicbIeJGwsQfgvW5szaqPDwLLkz/MMHuq1qBnnFHgfnlJrOVj4emfM7cSGRnaKGUXCGEHRO4XwW9PwllZSRPsHRPUR25R/jSEeAlrl1fWXSx3mf1xUMekpyQgbMsP3klzAxYUpDW59Y0nxq8uLu2r0hIp0MsUPNRVSMIODHDHMNQJiQUlqC+CrQL3qCiInKq79hMcCkuZnfWxsj+0yCUyPDVvl7Zew3C87H58V9zPCcYgYlIR1fJcwxHDxIo51stDSca+xLItXZHhXcn8ByP+iRj7PY9vB8jal8j9DitWP+fh/l5GbeUibTjyHdNiNdjC8nat3YOre2vQOpo5bYJB1qSjgAsDgmCASzkRRy/M21279zImCTf/LXAERfxMXsLC49C8UAUUbMikTUzdIHLgTMH3ycOFyDXvDLGju6CrqMLvTO6MfnqS7AuZpjvtkvIHAhNjiTyMiZGv30hFbrkZjKX5LKXMcVG0FqHYGlc+ZSxoQ6tVGMkJgLxgxS2BdMgK+i06X7s0rdvEflicE9ZUOISSJhxE1c8r5f8HAUqMc/Su4IdebWCay6q9kLzWr8k3zcXfAoJzr6W1zR+CMlPgcX2iY/L0teQDKacxkJOKOVY3xYAK/+nXzPno/Kta95cRv6FDtaLVm1LwLJv5gZfynlvZ0XngvszuA/PhlXlht8XRgj3qbjtf5MGy3NXcDpXBjlb7r3E9sGbON4lGtfuZdvMOjaweV2Dav9dbCZNBhlFZ3+z8CvZc1pzTaIXdD7rurLv+39i8hPKM/66GKlndvVIKyuh0l/DDR/i5N/LZ/9DAShkHNTxHhXxXq+Zr1QHTxSrC3BQIKDZvrYO1KU9gsgGR802kzy7J0IrvBqYrhrvOmrQAfgcVQK2tkWtterY2TTJLvOtHWCmf/0pff0Fi0=\"}"},9],"retry":0,"queue":"medium","backtrace":15,"jid":"f115da99c538161a2a36a7bd","created_at":1541046831.9780283,"locale":"en","enqueued_at":1541050308.8521712},"jobstr":"{\"class\":\"Workers::SendPrivate\",\"args\":[2,\"StatusMessage:59f0edb0bfa501368c061697a07b37d3\",{\"https://squeet.me/receive/users/xxxxxxxxxxxxxx\":\"{\\\"aes_key\\\":\\\"KI6Vc7ygfkIFl+vvSiOQorGUbqkePzurOXeXWElgF+4x+GS7dQq/vLCdeNvuCogDNGRVnLb6gyXu5NzfhdEOIOnKa4S+/8IvdQGScyPPkG54d1CdOUTaiwqp1LpWON7EC5AFzK6uUso4Y+nhJugY52wfA/l0qr0YE4NNeE0/SoijVj2nSyvQqBfLQ4htvOqCxthiMZ5Hp/89UzB/2PqnsxFmHhTDKSrcCpzD6fYiSnHzKRhSN8RwRvfB2baDNd8/rmjCc/afLMnNtJBsOu+BqpVoeo4Dt6JyIQNxVHe0Cd/nPt3fOKFm0IQPJ2CRAQiYnMxAyLzHTipoc5nNp7mp7YhMkqxVoo6qIR1rfHfvSffxPazAq6ZyZqpxZfD3bE1xxOLHmvQjh3QrtcPqb5u+3rXTkY9+zm54oc5Kq3AsQhokkXC5PWWNkj8yW4y1NwW+SGZbx4HrRnM3U4dYir+KwB2/f19rWrjY2hiSO7sSHhYgEnfMLMwd0zEaxi6mu9D+nX9YBi334+tMgB63dhUndiTg68D59weZGtzl7lN8pbLComIUBJO7CvgyRKwlm3P2YdvtQycOEyylPDtJt4+XWmRHcQDVqif1qyWm4E8lGigfUcoSjP1vnMcrs7vznXe6dVMcMxHm/3ZmubeH9eAc/7Ak+WJK5SpmO239l3NpLqI=\\\",\\\"encrypted_magic_envelope\\\":\\\"JER30XbKPaIUl60E5HgodvPGIPvjm+yI9HaF2PPcz/Bcql78zlZiSgDDXjMaBFUebSPlj7vGt9IJOaBTvyDVBewv5PiTj7kzCPKXGBtGnyHjzBcyOGQ3uNjL7WViAfeMHg1hdZiisdaaHOA2R29GsnSvL2dEZCdC54zWTdvuXOpb0ZgCXWqoZSjpXhjhQb7hmr74xxVtnhAh4Zt/583y9wv1OuvSa/wjJXlHr/fa0PE1TpJBROZhF8gTPsBxIh+FL4j9dakOjpDAdyQzXgz3KTANgmYo7CAgzNV8Js6yoH6wDFS3yRrUjFW96imZie4BZhoR9fbul6+/IgqjjoEPt8OJDOPqOkAgY+It1n1Y/Kgz2AeEzBWfwozXXQ5qsv5QKvLjfwDa8/LfIrsuORgjdSXjYCQqfimsVYJoSIZ340Ga2NsmyaBrAbHBbQKXyNQPEkK8kAErahIx/wzKlzSmdkgr4DyI7fszIbdp0Gh2vqLJnYQi7p9XOpwlwpikdZzByCfMukJMzWtWX2DTvcLcMYTIglekTF94wnG4gME7ZL1/Ue3J0jBCPS1cQ6oMBgHhHZVl1lzV/9jpg2e2i4MWiOzYnq6ARbWGr/1uyZollDEAglF89YPhjTICjE0wKZswzRW/ONM2DCgilV39waxXxCR31n/8YSzIYhDl+DDr/zsyUkQZYRMqusDKMlyjtSPSeXt8i7G3RtusyMfBIlfOyU3iNvlhSlSwc+g1qhOIQKRM82QaCiHB1zz/fSWcEzNSzC5KK7xYLIUGKcnNSJaMkLKmHxnqhZXVxge1Ol9OEYxkLCr8g2ELO4dS4LsokThN5q0eZDbuoX58/Z0jvH4OdZMLG4S3+FaKHxZtvk/NFQRGR2lWO2O9bXIjcQxxjRn+NJW1r3o0e6A3locjY4WOpV2q/pTHsMBZDqkofd6LAF36KK71Bm3BV6/2L4tJkJo1XO7hM6Gj6allH4tB0uIgIwIJhUBe2DzZPrquWswzais/2DYShiKkqQdL7AP8vfuH7e1zDvlQpg3pJ4f3jFWevHE5QAQk/4ZPvtlLyrGIw4WkTCGre2r0qVkf0HOoJgNDp/+e9EeJnHuDjcZi+f6OYlylCpunHHUsjDFff4RIfOqZUIkcZP7Wfx/71lP99TodtH1q1SIABAkWHcFEsaGgFsicbIeJGwsQfgvW5szaqPDwLLkz/MMHuq1qBnnFHgfnlJrOVj4emfM7cSGRnaKGUXCGEHRO4XwW9PwllZSRPsHRPUR25R/jSEeAlrl1fWXSx3mf1xUMekpyQgbMsP3klzAxYUpDW59Y0nxq8uLu2r0hIp0MsUPNRVSMIODHDHMNQJiQUlqC+CrQL3qCiInKq79hMcCkuZnfWxsj+0yCUyPDVvl7Zew3C87H58V9zPCcYgYlIR1fJcwxHDxIo51stDSca+xLItXZHhXcn8ByP+iRj7PY9vB8jal8j9DitWP+fh/l5GbeUibTjyHdNiNdjC8nat3YOre2vQOpo5bYJB1qSjgAsDgmCASzkRRy/M21279zImCTf/LXAERfxMXsLC49C8UAUUbMikTUzdIHLgTMH3ycOFyDXvDLGju6CrqMLvTO6MfnqS7AuZpjvtkvIHAhNjiTyMiZGv30hFbrkZjKX5LKXMcVG0FqHYGlc+ZSxoQ6tVGMkJgLxgxS2BdMgK+i06X7s0rdvEflicE9ZUOISSJhxE1c8r5f8HAUqMc/Su4IdebWCay6q9kLzWr8k3zcXfAoJzr6W1zR+CMlPgcX2iY/L0teQDKacxkJOKOVY3xYAK/+nXzPno/Kta95cRv6FDtaLVm1LwLJv5gZfynlvZ0XngvszuA/PhlXlht8XRgj3qbjtf5MGy3NXcDpXBjlb7r3E9sGbON4lGtfuZdvMOjaweV2Dav9dbCZNBhlFZ3+z8CvZc1pzTaIXdD7rurLv+39i8hPKM/66GKlndvVIKyuh0l/DDR/i5N/LZ/9DAShkHNTxHhXxXq+Zr1QHTxSrC3BQIKDZvrYO1KU9gsgGR802kzy7J0IrvBqYrhrvOmrQAfgcVQK2tkWtterY2TTJLvOtHWCmf/0pff0Fi0=\\\"}\"},9],\"retry\":0,\"queue\":\"medium\",\"backtrace\":15,\"jid\":\"f115da99c538161a2a36a7bd\",\"created_at\":1541046831.9780283,\"locale\":\"en\",\"enqueued_at\":1541050308.8521712}"}
Can anyone help me understand what’s going on here? If necessary I can post additional information from my log files. My concerns are that this is either (1) a code bug, (2) a configuration error on my pod or the other pod, or (3) a problem with that particular user’s account on my pod. I don’t have any idea how widespread this is, but I figure if “likes” have a problem, comments are also likely to fail, and this can lead to frustration for the impacted users.
Also, I should add that this problem relates to an aspected post that I created recently. I don’t know if the same thing might happen with a public post.
Thank you.
(NOTE: Replaced user guid with ‘xxxxxxxxxxxxxx’)