2020年12月30日 星期三

[研究]更新 Splunk 8.1.1 後,發現 log 沒進來之解決 (Index 爆滿)

[研究]更新 Splunk 8.1.1 後,發現 log 沒進來之解決 (Index 爆滿)

2020-12-30

更新 Splunk 8.1.1 後,發現 log 沒進來。

df 發現磁碟空間 99%。


參考

https://docs.splunk.com/Documentation/Splunk/8.1.1/Indexer/Setaretirementandarchivingpolicy
https://docs.splunk.com/Documentation/Splunk/9.0.1/Admin/Indexesconf


發現舊設定 /opt/splunk/etc/system/local/indexs.conf 不見了,可能升級整個目錄被砍掉重新安裝。

把 /opt/splunk/etc/system/default/indexs.conf 複製過來,設定可寫入,再次把預設值 6 年重新把值設回180天。(因為磁碟空間有限)


cp   /opt/splunk/etc/system/default/indexes.conf    /opt/splunk/etc/system/local/indexes.conf

chmod   a+w   /opt/splunk/etc/system/local/indexes.conf

vi   /opt/splunk/etc/system/local/indexes.conf


[main]

frozenTimePeriodInSecs = 15552000


預設值188697600秒/60/60/24/365=6年

新設值15552000秒/60/60/24=180天


reboot 作業系統 Linux 後,log又進來了。df 看到也不是99%了。

********************************************************************************

補:indexes.conf 完整內容


[default]
sync = 0

memPoolMB = auto

defaultDatabase = main

enableRealtimeSearch = true

suppressBannerList =

maxRunningProcessGroups = 8

maxRunningProcessGroupsLowPriority = 1

bucketRebuildMemoryHint = auto

serviceOnlyAsNeeded = true

serviceSubtaskTimingPeriod = 30

serviceInactiveIndexesPeriod = 60

maxBucketSizeCacheEntries = 0

processTrackerServiceInterval = 1

hotBucketTimeRefreshInterval = 10

rtRouterThreads = 0

rtRouterQueueSize = 10000

selfStorageThreads = 2

fileSystemExecutorWorkers = 5

hotBucketStreaming.extraBucketBuildingCmdlineArgs =

maxDataSize = auto

maxWarmDBCount = 300

frozenTimePeriodInSecs = 15552000

rotatePeriodInSecs = 60

coldToFrozenScript =

coldToFrozenDir =

compressRawdata = true

maxTotalDataSizeMB = 500000

maxGlobalRawDataSizeMB = 0

maxGlobalDataSizeMB = 0

maxConcurrentOptimizes = 6

maxHotSpanSecs = 7776000

maxHotIdleSecs = 0

maxHotBuckets = auto

metric.maxHotBuckets = auto

minHotIdleSecsBeforeForceRoll = auto

quarantinePastSecs = 77760000

quarantineFutureSecs = 2592000

rawChunkSizeBytes = 131072

minRawFileSyncSecs = disable

assureUTF8 = false

serviceMetaPeriod = 25

partialServiceMetaPeriod = 0

throttleCheckPeriod = 15

syncMeta = true

maxMetaEntries = 1000000

maxBloomBackfillBucketAge = 30d

enableOnlineBucketRepair = true

enableDataIntegrityControl = false

maxTimeUnreplicatedWithAcks = 60

maxTimeUnreplicatedNoAcks = 300

minStreamGroupQueueSize = 2000

warmToColdScript =

tstatsHomePath = volume:_splunk_summaries/$_index_name/datamodel_summary

homePath.maxDataSizeMB = 0

coldPath.maxDataSizeMB = 0

streamingTargetTsidxSyncPeriodMsec = 5000

journalCompression = gzip

enableTsidxReduction = false

suspendHotRollByDeleteQuery = false

tsidxReductionCheckPeriodInSec = 600

timePeriodInSecBeforeTsidxReduction = 604800

datatype = event

splitByIndexKeys =

metric.splitByIndexKeys =

tsidxWritingLevel = 1

archiver.enableDataArchive = false

archiver.maxDataArchiveRetentionPeriod = 0

hotBucketStreaming.sendSlices = false

hotBucketStreaming.removeRemoteSlicesOnRoll = false

hotBucketStreaming.reportStatus = false

hotBucketStreaming.deleteHotsAfterRestart = false

tsidxTargetSizeMB = 1500

metric.tsidxTargetSizeMB = 1500

metric.enableFloatingPointCompression = true

metric.compressionBlockSize = 1024

metric.stubOutRawdataJournal = true

metric.timestampResolution = s

waitPeriodInSecsForManifestWrite = 60

repFactor = 0

[_audit]
homePath = $SPLUNK_DB/audit/db
coldPath = $SPLUNK_DB/audit/colddb
thawedPath = $SPLUNK_DB/audit/thaweddb
tstatsHomePath = volume:_splunk_summaries/audit/datamodel_summary

[_internal]
homePath = $SPLUNK_DB/_internaldb/db
coldPath = $SPLUNK_DB/_internaldb/colddb
thawedPath = $SPLUNK_DB/_internaldb/thaweddb
tstatsHomePath = volume:_splunk_summaries/_internaldb/datamodel_summary
maxDataSize = 1000
maxHotSpanSecs = 432000
frozenTimePeriodInSecs = 2592000

[_introspection]
homePath = $SPLUNK_DB/_introspection/db
coldPath = $SPLUNK_DB/_introspection/colddb
thawedPath = $SPLUNK_DB/_introspection/thaweddb
maxDataSize = 1024
frozenTimePeriodInSecs = 1209600

[_metrics]
homePath = $SPLUNK_DB/_metrics/db
coldPath = $SPLUNK_DB/_metrics/colddb
thawedPath = $SPLUNK_DB/_metrics/thaweddb
datatype = metric
frozenTimePeriodInSecs = 1209600
metric.splitByIndexKeys = metric_name

[_metrics_rollup]
homePath = $SPLUNK_DB/_metrics_rollup/db
coldPath = $SPLUNK_DB/_metrics_rollup/colddb
thawedPath = $SPLUNK_DB/_metrics_rollup/thaweddb
datatype = metric
frozenTimePeriodInSecs = 63072000
metric.splitByIndexKeys = metric_name

[_telemetry]
homePath = $SPLUNK_DB/_telemetry/db
coldPath = $SPLUNK_DB/_telemetry/colddb
thawedPath = $SPLUNK_DB/_telemetry/thaweddb
maxDataSize = 256
frozenTimePeriodInSecs = 63072000

[_thefishbucket]
homePath = $SPLUNK_DB/fishbucket/db
coldPath = $SPLUNK_DB/fishbucket/colddb
thawedPath = $SPLUNK_DB/fishbucket/thaweddb
tstatsHomePath = volume:_splunk_summaries/fishbucket/datamodel_summary
maxDataSize = 500
frozenTimePeriodInSecs = 2419200

[history]
homePath = $SPLUNK_DB/historydb/db
coldPath = $SPLUNK_DB/historydb/colddb
thawedPath = $SPLUNK_DB/historydb/thaweddb
tstatsHomePath = volume:_splunk_summaries/historydb/datamodel_summary
maxDataSize = 10
frozenTimePeriodInSecs = 604800

[main]
homePath = $SPLUNK_DB/defaultdb/db
coldPath = $SPLUNK_DB/defaultdb/colddb
thawedPath = $SPLUNK_DB/defaultdb/thaweddb
tstatsHomePath = volume:_splunk_summaries/defaultdb/datamodel_summary
maxConcurrentOptimizes = 6
maxHotIdleSecs = 86400
maxHotBuckets = 10
maxDataSize = auto_high_volume

[provider-family:hadoop]
vix.mode = report
vix.command = $SPLUNK_HOME/bin/jars/sudobash
vix.command.arg.1 = $HADOOP_HOME/bin/hadoop
vix.command.arg.2 = jar
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-h1.jar
vix.command.arg.4 = com.splunk.mr.SplunkMR
vix.env.MAPREDUCE_USER =
vix.env.HADOOP_HEAPSIZE = 512
vix.env.HADOOP_CLIENT_OPTS = -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.19.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.10.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-exec-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-metastore-0.12.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive/hive-serde-0.12.0.jar
vix.mapred.job.reuse.jvm.num.tasks = 100
vix.mapred.child.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapred.reduce.tasks = 0
vix.mapred.job.map.memory.mb = 2048
vix.mapred.job.reduce.memory.mb = 512
vix.mapred.job.queue.name = default
vix.mapreduce.job.jvm.numtasks = 100
vix.mapreduce.map.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapreduce.reduce.java.opts = -server -Xmx512m -XX:ParallelGCThreads=4 -XX:+UseParallelGC -XX:+DisplayVMOutputToStderr
vix.mapreduce.job.reduces = 0
vix.mapreduce.map.memory.mb = 2048
vix.mapreduce.reduce.memory.mb = 512
vix.mapreduce.job.queuename = default
vix.splunk.search.column.filter = 1
vix.splunk.search.mixedmode = 1
vix.splunk.search.debug = 0
vix.splunk.search.mr.maxsplits = 10000
vix.splunk.search.mr.minsplits = 100
vix.splunk.search.mr.splits.multiplier = 10
vix.splunk.search.mr.poll = 2000
vix.splunk.search.recordreader = SplunkJournalRecordReader,ValueAvroRecordReader,SimpleCSVRecordReader,SequenceFileRecordReader
vix.splunk.search.recordreader.avro.regex = \.avro$
vix.splunk.search.recordreader.csv.regex = \.([tc]sv)(?:\.(?:gz|bz2|snappy))?$
vix.splunk.search.recordreader.sequence.regex = \.seq$
vix.splunk.home.datanode = /tmp/splunk/$SPLUNK_SERVER_NAME/
vix.splunk.heartbeat = 1
vix.splunk.heartbeat.threshold = 60
vix.splunk.heartbeat.interval = 1000
vix.splunk.setup.onsearch = 1
vix.splunk.setup.package = current

[splunklogger]
homePath = $SPLUNK_DB/splunklogger/db
coldPath = $SPLUNK_DB/splunklogger/colddb
thawedPath = $SPLUNK_DB/splunklogger/thaweddb
disabled = true

[summary]
homePath = $SPLUNK_DB/summarydb/db
coldPath = $SPLUNK_DB/summarydb/colddb
thawedPath = $SPLUNK_DB/summarydb/thaweddb
tstatsHomePath = volume:_splunk_summaries/summarydb/datamodel_summary

[volume:_splunk_summaries]
path = $SPLUNK_DB


不清楚 $SPLUNK_HOME 和 $SPLUNK_DB 是甚麼值?

[root@aplog local]# echo $SPLUNK_DB

[root@aplog local]# echo $SPLUNK_HOME

[root@aplog local]#


cat /opt/splunk/etc/splunk-launch.conf.default

#   Version 8.2.6

# Modify the following line to suit the location of your Splunk install.
# If unset, Splunk will use the parent of the directory containing the splunk
# CLI executable.
#
# SPLUNK_HOME=/opt/splunk-home

# By default, Splunk stores its indexes under SPLUNK_HOME in the
# var/lib/splunk subdirectory.  This can be overridden
# here:
#
# SPLUNK_DB=/opt/splunk-home/var/lib/splunk
# Splunkd daemon name
SPLUNK_SERVER_NAME=Splunkd

# If SPLUNK_OS_USER is set, then Splunk service will only start
# if the 'splunk [re]start [splunkd]' command is invoked by a user who
# is, or can effectively become via setuid(2), $SPLUNK_OS_USER.
# (This setting can be specified as username or as UID.)
#
# SPLUNK_OS_USER

cat /opt/splunk/etc/splunk-launch.conf

# Copyright (C) 2005-2011 Splunk Inc. All Rights Reserved.  Version 4.2.3

# Modify the following line to suit the location of your Splunk install.
# If unset, Splunk will use the parent of the directory this configuration
# file was found in
#
SPLUNK_HOME=/opt/splunk

# By default, Splunk stores its indexes under SPLUNK_HOME in the
# var/lib/splunk subdirectory.  This can be overridden
# here:
#
# SPLUNK_DB=/opt/splunk/var/lib/splunk

# Splunkd daemon name
SPLUNK_SERVER_NAME=splunkd

# Splunkweb daemon name
SPLUNK_WEB_NAME=splunkweb


所以實際上 index DB 存放在 SPLUNK_DB=/opt/splunk/var/lib/splunk

(完)

[研究] CentOS 6.10 的 yum 疑似不能更新、安裝、移除套件了

[研究] CentOS 6.10 的 yum 疑似不能更新、安裝、移除套件了

2020-12-30

CentOS 6.10是2018-07-03 釋出,是 CentOS 6.x 最後一版,早已不提供下載,也不提供更新,因為某些緣故,這台不能重新安裝,也無法另找機器安裝CentOS 7.x 或 8.x。



(下圖) Cannot find a valid baseurl for repo:base




(完)

相關

CentOS - 維基百科,自由的百科全書

https://zh.wikipedia.org/wiki/CentOS

zh-tw/Download - CentOS Wiki

https://wiki.centos.org/zh-tw/Download


2020年12月28日 星期一

[研究][ASP.NET]無法載入 Viewstate。Viewstate 所要載入的控制項樹狀結構必須符合在先前要求期間用來儲存 Viewstate 的控制項樹狀結構。

[研究][ASP.NET]無法載入 Viewstate。Viewstate 所要載入的控制項樹狀結構必須符合在先前要求期間用來儲存 Viewstate 的控制項樹狀結構。

2020-12-28

Visual Studio 2019 v16.8.3 + .NET Framework 4.7.2 + ASP.NET + WebForm

無法載入 Viewstate。Viewstate 所要載入的控制項樹狀結構必須符合在先前要求期間用來儲存 Viewstate 的控制項樹狀結構。例如,以動態方式加入控制項時,在回傳期間加入的控制項必須符合在初始要求期間所加入控制項的型別和位置。

(下圖) Click 圖片可看 100% 原始尺寸


其中

<asp:GridView ID="GridView1"... (略)
    <Columns>
        <asp:TemplateField ShowHeader="False">
            <ItemTemplate>
                <asp:LinkButton ID="LinkButton5" runat="server" CausesValidation="False" 
   		            CommandName="Edit2" Text="修改" 
					CommandArgument='<%# Eval("SN") %>' CssClass="btn btn-primary btn-xs"></asp:LinkButton>
            </ItemTemplate>
        </asp:TemplateField>

protected void GridView1_RowCommand(object sender, GridViewCommandEventArgs e)
{
    if (e.CommandName == "Edit2")
    {
        PlaceHolder1.Visible = false;
        GridView1.Visible = false;
		
        string id = e.CommandArgument.ToString();
        SqlDataSource2.SelectParameters["SN"].DefaultValue = id;
        
        DetailsView1.Visible = true;
        DetailsView1.Caption = "編輯";
        DetailsView1.ChangeMode(DetailsViewMode.Edit);
    }
}

研究後,將

protected void DetailsView1_ItemUpdated(object sender, DetailsViewUpdatedEventArgs e)
{
    PlaceHolder1.Visible = true;    // 新增按鈕
    GridView1.Visible = true;       // 列表畫面
    DetailsView1.Visible = false;   // 關閉 新增/修改 用的畫面

    // 更新 GridView1
    GridView1.DataSourceID = "";
    GridView1.DataSourceID = "SqlDataSource1";
    GridView1.DataBind();
}

改為

protected void DetailsView1_ItemUpdated(object sender, DetailsViewUpdatedEventArgs e)
{
    Response.Redirect(Request.RawUrl);
    //Response.Redirect(Request.Url.PathAndQuery);
}

解決。

(完)


[研究][ASP.NET]另用按鈕切換 DetailsView 模式

[研究][ASP.NET][WebForm] 另用按鈕切換 DetailsView 模式

2020-12-28

Visual Studio 2019 v16.8.3 + ASP.NET + WebForm

DetailsView1 下方本有 編輯、刪除、新增 等按紐可用,但若要另外用按紐去切換。 

********************************************************************************
失敗
protected void Button1_Click(object sender, EventArgs e)
        {
            DetailsView1.DefaultMode = DetailsViewMode.Insert;
        }
********************************************************************************
失敗
protected void Button1_Click(object sender, EventArgs e)
        {
            DetailsView1.DefaultMode = DetailsViewMode.Insert;
            DetailsView1.DataBind();
        }
********************************************************************************
成功
protected void Button1_Click(object sender, EventArgs e)
{
    DetailsView1.ChangeMode(DetailsViewMode.Insert);
}
protected void Button2_Click(object sender, EventArgs e)
{
    DetailsView1.ChangeMode(DetailsViewMode.Edit);
}
protected void Button3_Click(object sender, EventArgs e)
{
    DetailsView1.ChangeMode(DetailsViewMode.ReadOnly);
}

********************************************************************************

(完)

2020年12月26日 星期六

[研究]Windows Server 2019安裝Microsoft Edge (Chrome, Chromium 核心)

[研究]Windows Server 2019安裝Microsoft Edge (Chrome, Chromium 核心)

2020-12-26
2021-06-28 補充

微軟推出了 Chrome 核心的 Microsoft Edge瀏覽器,Windows 10 上安裝輕易,但發現 Windows Server 2019上要安裝似乎沒有那麼容易。

(下圖)

下載新版 Microsoft Edge 瀏覽器 | Microsoft

https://www.microsoft.com/zh-tw/edge




Windows Server 2019 預設沒有 Edge,但這個網頁要求 Edge 才能開啟。
從 IE11 改用 Chrome,結果還是相同。

(下圖) 相同網址,往下捲,按下「立即試用」,結果相同。

********************************************************************************
(下圖) 換網址下載,結果也失敗


********************************************************************************
(下圖) 換網址,也是不能下載,按鈕不能點



********************************************************************************
(下圖) 查需求

Microsoft Edge 支援的作業系統 | Microsoft Docs

https://docs.microsoft.com/zh-tw/deployedge/microsoft-edge-supported-operating-systems

Windows Server 2019 (LTSC) <= 要 LTSC 版本(Long Time Service Channel,長期服務通道)才行



********************************************************************************
(下圖) 靠 Google 大神,下載到線上安裝版,和離線安裝版 (請自己找)
MicrosoftEdgeSetup.exe 線上安裝檔案 

x64  ( MicrosoftEdgeEnterpriseX64.msi )

x86 ( MicrosoftEdgeEnterpriseX86.msi )

MacOS  ( MicrosoftEdgeEnterpriseARM64.msi )






在非 LTSC 版的 Windows Server 2019 上安裝成功了,也可以更新,想回官方去下載,但沒得下載,因為已經安裝了。


********************************************************************************

2021-06-28 補充

Windows Server 2019 有官方提供 Edge 安裝了。

[研究]Windows Server 2019安裝 Microsoft Edge 瀏覽器 (Chrome, Chromium 核心)(二)
https://shaurong.blogspot.com/2021/06/windows-server-2019-microsoft-edge.html

(完)

[研究]已在 Microsoft Exchange Server 上暫時移動您的信箱

[研究]已在 Microsoft Exchange Server 上暫時移動您的信箱

2020-12-26

( 轉貼請附上來源網址,文章不定期更新)

啟動家中電腦的 Outlook 2016,出現錯誤訊息。

已在 Microsoft Exchange Server 上暫時移動您的信箱。有一個暫存信箱存在,但是不一定保有您所有的舊資料。

您的郵箱已臨時在 Microsoft Exchange 伺服器上移動。 存在臨時郵箱,但可能沒有您以前的所有資料。 您可以連線到臨存信箱,或離線使用您所有的舊資料。如果您選擇使用舊資料,則無法傳送或接收電子郵件訊息。


之後,啟動位於公司電腦 Outlook 2016,並不會這樣。開始同時操作兩地的電腦測試,嘗試解決。

註:因為怕休假或下班後訊息晚看到,加上信箱有容量限制,而且 WebMail 對某些加密、加簽信件有所不便,敝人習慣2地電腦都用 Outlook 把信件複製收下一份。而且自己電腦上硬碟容量大,可以長期存信)

(只要設定信件不收下,用電子郵件規則在收到Email複製一份到另一地即可,信箱容量不夠時,可以把某些信刪除,反正已經複製一份走了)

********************************************************************************

Click 圖片可以看 100% 原圖。

(下圖)先看看「使用暫存信箱」





(下圖)雖說出現 Office 更新,Windows Update並無出現更新,且 Outlook 關閉,啟動,數次都沒有任何更新Office 或Outlook軟體狀況發生。






(下圖)想刪除信箱資料檔,讓他重建,失敗






(下圖) 重建一個相同的信箱看看



想起公司 WebMail 對外昨天關閉了,以後只能在公司內 or 先 SVPN 連上才能用。
後來測試用瀏覽器 SVPN 先連上登入WebMail,再啟動 Outlook,但 Outlook 這邊情況依舊。

( 因為 有駭客用 WebMail 猜密碼,而公司政策是錯誤 N 次會鎖住帳號若干時間,結果 Windows 無法登入 AD 網域,一堆事情不能做,最後封閉外部直接使用 WebMail )

(下圖) 寄信測試




(下圖) 測試「使用舊資料」





最後情況舊是家中 Outlook 每次啟動,都選「使用舊資料」,可以寄信,但無法收信,收信要用 SVPN + WebMail。

在公司,正常依舊。

(完)

相關

當 Office 365 使用者打開 Outlook 時,"您的郵箱已臨時在 Microsoft Exchange 伺服器上移動"消息適用於: Exchange OnlineOutlook 2016Outlook for Office 365 更多

https://support.microsoft.com/zh-tw/help/3197025/your-mailbox-has-been-temporarily-moved-on-microsoft-exchange-server

若要 Exchange Server 的信箱移動將會導致 Outlook 連線問題

https://support.microsoft.com/zh-tw/help/2934750/mailbox-move-to-exchange-server-causes-outlook-connectivity-issue


2020年12月25日 星期五