BIhints - spilling the beans since 2006

Beware of Performance Monitor defaults spamming transaction log

admin — Thu, 17 Aug 2023 16:46:31 +0000

Beware of Performance Monitor defaults spamming transaction log

The Performance Monitor in TM1 is a great tool to gather helpful statistics about your system and pinpoint all sorts of bottlenecks.

You can start the Performance Monitor in 2 ways:

In Server Explorer -> Server -> Start Performance Monitor
add PerformanceMonitorOn=T in tm1s.cfg, this is a dynamic parameter, it will come into action within 60 seconds

Once performance monitor is turned on, it starts populating the }Stats control cubes and it updates them every minute.
Well, so far so good...
BUT by default in TM1, ALL cubes in }CubeProperties are set to log ALL transactions. That seems like a sane default, nobody wants to lose track of whom, when and by how much changed a cell in a cube.
So by default, the Performance Monitor will start spamming the transaction log, every minute of your entire server uptime. The problem is most acute with }StatsByCube, it logs one line for every cube, and for every }Element_Attributes_dimension cube.

There is a simple remedy: turn off logging in }CubeProperties for all }Stats cubes (IBM documentation is incomplete)
A quick Turbo Integrator process can do the job too:

CellPutS('NO', '}CubeProperties', '}StatsByClient',       'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsByCube',         'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsByCubeByClient', 'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsByRule',         'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsByProcess',      'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsByChore',        'LOGGING');
CellPutS('NO', '}CubeProperties', '}StatsForServer',      'LOGGING');

The size of the spamming could be just a few megabytes per day, though it will have an impact on how long your transaction searches take.
Also, the transaction log does not log rotate for a good reason, you don't want to lose any precious transactions. So, it will happily fill up the local drive and after a few months or years the server finally crashes.
If you notice the issue only now, it's not all lost. You don't have to keep all this spam thanks to this bash one-liner that will clean up your 50 gigabyte logs:

for f in tm1s20*log; do awk -F, '$8 != "\"}StatsByCube\""' "$f" > "$f".tmp && mv "$f".tmp "$f"; done # one line to rule them all

You will notice it is removing only "}StatsByCube". I could have added the other }StatsBy cubes, but these are literally 2 orders of magnitude less spammy than }StatsByCube on its own.

Challenge for you: try to achieve the same as the bash one-liner above in any language in less than the 79 characters above. I made it easy, I left a lot of superfluous characters. Constructive comments and answers are welcome.

admin Thu, 2023-08-17 18:46

Categories

monitor

admins

Add new comment

tm1s.cfg parameters cheatsheet

admin — Tue, 16 May 2023 12:10:44 +0000

tm1s.cfg parameters cheatsheet

The number of parameters in the tm1s.cfg has been steadily rising as more features have been made available over the years in TM1. So, a cheatsheet summarising all these parameters can help as an overview of the different settings that a TM1 server can be configured with.

This cheatsheet is now available in the following formats:

csv so you can build your own variations of the cheatsheet
tm1s.cfg including all parameters, only the required parameters are uncommented, every parameter is set to its known default value (as of 2023-02-14 and according to the IBM online documentation). There should not be a need to explicitly define a parameter when you just want to stick to the default value. However, be cautious that IBM may change default values and that could lead to issues when upgrading versions.
md markdown to save in your personal knowledge management repository full of markdown files to ripgrep through fzf.vim
pdf for an allegedly pretty overview (coming soon)
html below or here on its own that you can "Save as" to use as a local copy.

The cheatsheet is generated with python scripts (coming soon) responsible for the conversion of the csv file to a pandas dataframe and then exported to different formats. The csv file currently holds the following fields:

parameter: name of the parameter
optional: indicates if the parameter is either required or optional.
restart: indicates if the parameter is either dynamic or static 🔄, in the later case, it requires a service restart when its value has been modified
url: URL address of the official IBM Planning Analytics documentation for that parameter
default: default value of that parameter when it is not explicitly declared
type: type of that parameter (bool, int, float, str, time). This will be useful for scripts checking tm1s.cfg validity
context: arbitrary categorisation for the context of the parameter e.g. it can be Network related, Rules related, etc..
dependency: that may hold the name of another parameter that the current parameter may rely on to operate
description: short description about what the parameter does

The categorisation is completely arbitrary, most parameters could fit in several categories, so I just picked one that seemed most appropriate. This is a limitation and probably something to look into for the design of future versions of that csv file.

And finally, the tm1s.cfg parameters cheatsheet in html.

Note that there is a drop down menu at the top of the Context column. This allows you to show only parameters that relate to a specific aspect of your TM1 server. If you are on mobile, you probably need to rotate your screen for a wide display in order to display all 5 columns.

tm1s.cfg parameters cheatsheet

parameters in bold are required.
🔄 = static parameter, the TM1 service must be restarted for the new value of that parameter to take effect.

parameter	default	restart	context	description
AdminHost		🔄	Network	computer name or IP address of the Admin Host on which an Admin Server is running.
AllowReadOnlyChoreReschedule	F	🔄	Chores	Provides users with READ access to a chore and the ability to activate deactivate and reschedule chores.
AllowSeparateNandCRules	F	🔄	Rules	if true then rule expressions for N: and C: levels can be on separate lines using identical AREA definitions.
AllRuleCalcStargateOptimization	F	🔄	Rules	if true then it can improve performance in calculating views that contain only rule-calculated values. In some unique cases enabling this parameter can result in performance degradation so you should test the effect of the parameter in a development environment before deploying to your production environment.
ApplyMaximumViewSizeToEntireTransaction	F		Views	Applies MaximumViewSize to the entire transaction instead of to individual calculations.
AuditLogMaxFileSize	100 MB		Logs and Messages	maximum file size that an audit log file can grow to before it is closed and a new file is created.
AuditLogMaxQueryMemory	100 MB		Logs and Messages	maximum amount of memory that TM1 can use when running an audit log query and retrieving the set of results.
AuditLogOn	F		Logs and Messages	if true turns audit logging on.
AuditLogUpdateInterval	60		Logs and Messages	maximum amount of minutes that TM1 waits before moving the events from the temporary audit file into the final audit log.
AutomaticallyAddCubeDependencies	T	🔄	Rules	Determines if cube dependencies are set automatically or if you must manually identify the cube dependencies for each cube.
CacheFriendlyMalloc	F	🔄	Cache	Allows for memory alignment that is specific to the IBM Power Platform.
CalculationThresholdForStorage	50		Rules	minimum number of rule calculations required for a single cell or Stargate view beyond which TM1 stores the calculations for use during the current server session.
CAMPortalVariableFile		🔄	Cognos Analytics	The path to the variables_TM1.xml file in your IBM Cognos installation.
CAMUseSSL	F	🔄	Cognos Analytics	all communications between TM1 and the Cognos Analytics server must use SSL.
CheckCAMClientAlias	T		Cognos Analytics	When audit logging is enabled with the AuditLogOn parameter the CheckCAMClientAlias parameter determines whether user modifications within Cognos Authentication Manager (CAM) groups are written to the audit log.
CheckFeedersMaximumCells	3000000		Rules	Limits the number of cells checked by the Check Feeders option in the Cube Viewer.
ClientCAMURI			Cognos Analytics	The URI the Cognos Server Connection uses to authenticate TM1 clients.
ClientExportSSLSvrCert	F	🔄	Authentication and Encryption	TM1 client should retrieve the certificate authority certificate which was originally used to issue TM1 certificate from the MS Windows certificate store.
ClientExportSSLSvrKeyID		🔄	Authentication and Encryption	identity key used by a TM1 client to export the certificate authority certificate which was originally used to issue TM1 certificate from the MS Windows certificate store.
ClientMessagePortNumber		🔄	Network	secondary port used to accept client messages concerning the progress and ultimate cancellation of a lengthy operation without tying up thread reserves.
ClientPingCAMPassport	900		Cognos Analytics	interval in seconds that a client should ping the Cognos Authentication Management server to keep their passport alive.
ClientPropertiesSyncInterval			Logs and Messages	frequency (in seconds) at which client properties are updated in the }ClientProperties control cube. Set to 1800 seconds to update cube every 30 minutes.
ClientVersionMaximum			Network	maximum client version that can connect to TM1.
ClientVersionMinimum	8.4.00000		Network	minimum client version that can connect to TM1.
ClientVersionPrecision	0		Network	precisely identify the minimum and maximum versions of clients that can connect to TM1.
CognosMDX.AggregateByAncestorRef	F	🔄	Cognos Analytics	When possible replaces aggregation over a member set with a reference to an ancestor if the aggregated member set comprises a complete set of descendants and all members have the weight 1.
CognosMDX.CellCacheEnable	T	🔄	Cognos Analytics	Allows the IBM Cognos MDX engine to modify TM1 consolidation and calculation cell cache strategies.
CognosMDX.PrefilterWithPXJ	F	🔄	Cognos Analytics	Expands the data source provider cross join approach to nested filtered sets.
CognosMDX.SimpleCellsUseOPTSDK	T	🔄	Cognos Analytics	Applies IBM Cognos MDX engine consolidation and calculation cell cache strategies to all cells in query results.
CognosMDX.UseProviderCrossJoinThreshold	0	🔄	Cognos Analytics	Applies the data source provider cross join strategy even if it is not explicitly enabled in Cognos Analytics.
CognosTM1InterfacePath		🔄	Cognos Analytics	location of the IBM Cognos Analytics server to use when importing data from a Cognos Package to TM1 configuration using the Package Connector.
CreateNewCAMClients	T		Cognos Analytics	The CreateNewCAMClients server configuration parameter determines how TM1 configuration handles an attempt to log on to the server with CAM credentials in the absence of a corresponding TM1 client.
DataBaseDirectory		🔄	Files	data directory from which the database loads cubes dimensions and other objects.
DefaultMeasuresDimension	F	🔄	Cubes	Identifies if a measures dimension is created. TM1 does not require that a measures dimension be defined for a cube. You can optionally define a measures dimension by modifying the cube properties.
DisableMemoryCache	F	🔄	Cache	Disables the memory cache used by TM1 memory manager.
DisableSandboxing	F		Sandboxes	if false then users have the ability to use sandboxes across the server.
DownTime			Chores	time when the database will come down automatically.
EnableNewHierarchyCreation	F	🔄	Dimensions	multiple hierarchy creation is enabled or disabled.
EnableSandboxDimension	F		Sandboxes	virtual sandbox dimension feature is enabled or disabled.
EnableTIDebugging	F		Turbo Integrator	TurboIntegrator debugging capabilities are enabled or disabled.
EventLogging	T		Logs and Messages	event logger is either turned on or off.
EventScanFrequency	1		Logs and Messages	period to check the collection of threads where 1 is the minimum number and 0 disables the scan.
EventThreshold.PooledMemoryInMB	0		Logs and Messages	threshold for which a message is printed for the event that the database's pooled memory exceeds a certain value.
EventThreshold.ThreadBlockingNumber	5		Logs and Messages	warning is printed when a thread blocks at least the specified number of threads.
EventThreshold.ThreadRunningTime	600		Logs and Messages	warning is printed when a thread has been running for the specified length of time.
EventThreshold.ThreadWaitingTime	20		Logs and Messages	warning is printed when a thread has been blocked by another thread for the specified length of time.
ExcelWebPublishEnabled	F		TM1Web	Enables the publication of MS Excel files to IBM Cognos TM1 Web as well as the export of MS Excel files from TM1 Web when MS Excel is not installed on the web server. Enable the ExcelWebPublishEnabled parameter when you have TM1 10.1 clients that connect to TM1 10.2.2 servers.
FileRetry.Count	5		Logs and Messages	number of retry attempts.
FileRetry.Delay	2000		Logs and Messages	time delay between retry attempts.
FileRetry.FileSpec			Logs and Messages
FIPSOperationMode	2	🔄	Authentication and Encryption	Controls the level of support for Federal Information Processing Standards (FIPS).
ForceReevaluationOfFeedersForFedCellsOnDataChange	F	🔄	Rules	When this parameter is set a feeder statement is forced to be re-evaluated when data changes.
HTTPOriginAllowList			Network	a comma-delimited list of external origins (URLs) that are trusted and can access TM1.
HTTPPortNumber	5001	🔄	Network	port number on which TM1 listens for incoming HTTP(S) requests.
HTTPRequestEntityMaxSizeInKB	32		Network	This TM1 configuration parameter sets the maximum size for an HTTP request entity that can be handled by TM1.
HTTPSessionTimeoutMinutes	20		Authentication and Encryption	timeout value for authentication sessions for TM1 REST API.
IdleConnectionTimeOutSeconds			Authentication and Encryption	timeout limit for idle client connections in seconds.
IndexStoreDirectory		🔄	Files	Added in v2.0.5 Designates a folder to store index files including bookmark files.
IntegratedSecurityMode			Authentication and Encryption	the user authentication mode to be used by TM1.
IPAddressV4		🔄	Network	IPv4 address for an individual TM1 service.
IPAddressV6		🔄	Network	IPv6 address for an individual TM1 service.
IPVersion	ipv4	🔄	Network	Internet Protocol version used by TM1 to identify IP addresses on the network.
JavaClassPath		🔄	Java	parameter to make third-party Java™ libraries available to TM1.
JavaJVMArgs		🔄	Java	list of arguments to pass to the Java Virtual Machine (JVM). Arguments are separated by a space and the dash character. For example JavaJVMArgs=-argument1=xxx -argument2=yyy.
JavaJVMPath		🔄	Java	path to the Java Virtual Machine .dll file (jvm.dll) which is required to run Java from TM1 TurboIntegrator.
keyfile		🔄	Authentication and Encryption	file path to the key database file. The key database file contains the server certificate and trusted certificate authorities. The server certificate is used by TM1 and the Admin server.
keylabel		🔄	Authentication and Encryption	label of the server certificate in the key database file.
keystashfile		🔄	Authentication and Encryption	file path to the key database password file. The key database password file is the key store that contains the password to the key database file.
Language		🔄	Logs and Messages	language used for TM1. This parameter applies to messages generated by the server and is also used in the user interface of the server dialog box when you run the server as an application instead of a Windows service.
LDAPHost	localhost	🔄	Authentication and Encryption	domain name or dotted string representation of the IP address of the LDAP server host.
LDAPPasswordFile		🔄	Authentication and Encryption	password file used when LDAPUseServerAccount is not used. This is the full path of the .dat file that contains the encrypted password for the Admin Server's private key.
LDAPPasswordKeyFile		🔄	Authentication and Encryption	password key used when LDAPUseServerAccount is not used.
LDAPPort	389	🔄	Authentication and Encryption	port TM1 uses to bind to an LDAP server.
LDAPSearchBase		🔄	Authentication and Encryption	node in the LDAP tree where TM1 begins searching for valid users.
LDAPSearchField	cn	🔄	Authentication and Encryption	The name of the LDAP attribute that is expected to contain the name of TM1 user being validated.
LDAPSkipSSLCertVerification	F	🔄	Authentication and Encryption	Skips the certificate trust verification step for the SSL certificate used to authenticate to an LDAP server. This parameter is applicable only when LDAPVerifyServerSSLCert=T.
LDAPSkipSSLCRLVerification	F	🔄	Authentication and Encryption	Skips CRL checking for the SSL certificate used to authenticate to an LDAP server. This parameter is applicable only when LDAPVerifyServerSSLCert=T.
LDAPTimeout	0		Authentication and Encryption	number of seconds that TM1 waits to complete a bind to an LDAP server. If the LDAPTimeout value is exceeded TM1 immediately aborts the connection attempt.
LDAPUseServerAccount		🔄	Authentication and Encryption	Determines if a password is required to connect to TM1 when using LDAP authentication.
LDAPVerifyCertServerName	F	🔄	Authentication and Encryption	server to use during the SSL certificate verification process for LDAP server authentication. This parameter is applicable only when LDAPVerifyServerSSLCert=T.
LDAPVerifyServerSSLCert	F	🔄	Authentication and Encryption	Delegates the verification of the SSL certificate to TM1. This parameter is useful for example when you are using LDAP with a proxy server.
LDAPWellKnownUserName		🔄	Authentication and Encryption	user name used by TM1 to log in to LDAP and look up the name submitted by the user.
LoadPrivateSubsetsOnStartup	F	🔄	Subsets	This configuration parameter determines if private subsets are loaded when TM1 starts.
LoadPublicViewsAndSubsetsAtStartup	T	🔄	Views	Added in v2.0.8 This configuration parameter enables whether public subsets and views are loaded when the TM1 starts and keeps them loaded to avoid lock contention during the first use.
LockPagesInMemory	F	🔄	deprecated	Deprecated as of IBM TM1 with Watson version 2.0.9.7 When this parameter is enabled memory pages used by TM1 process are held in memory (locked) and do not page out to disk under any circumstances. This retains the pages in memory over an idle period making access to TM1 data faster after the idle period.
LoggingDirectory		🔄	Logs and Messages	directory to which TM1 saves its log files.
LogReleaseLineCount	5000		Logs and Messages	number of lines that a search of the Transaction Log will accumulate in a locked state before releasing temporarily so that other Transaction Log activity can proceed.
MagnitudeDifferenceToBeZero		🔄	Calculations	order of magnitude of the numerator relative to the denominator above which the denominator equals zero when using a safe division operator.
MaskUserNameInServerTools	T	🔄	Logs and Messages	Determines whether usernames in server administration tools are masked until a user is explicitly verified as having administrator access.
MaximumCubeLoadThreads	0	🔄	Multithreading	Specifies whether the cube load and feeder calculation phases of server loading are multi-threaded so multiple processor cores can be used in parallel.
MaximumLoginAttempts	3		Authentication and Encryption	maximum number of failed user login attempts permissible on TM1.
MaximumMemoryForSubsetUndo	10240		Cache	maximum amount of memory in kilobytes to be dedicated to storing the Undo/Redo stack for the Subset Editor.
MaximumSynchAttempts	1	🔄	Network	maximum number of times a synchronization process on a planet database will attempt to reconnect to a network before the process fails.
MaximumTILockObjects	2000	🔄	Turbo Integrator	This configuration parameter sets the maximum lock objects for a TurboIntegrator process. Used by the synchronized() TurboIntegrator function.
MaximumUserSandboxSize	500 MB		Sandboxes	maximum amount of RAM memory (in MB) to be allocated per user for personal workspaces or sandboxes.
MaximumViewSize	500 MB		Views	maximum amount of memory (in MB) to be allocated when a user accesses a view.
MDXSelectCalculatedMemberInputs	T		MDX	Changes the way in which calculated members in MDX expressions are handled when zero suppression is enabled.
MemoryCache.LockFree	F		Cache	Switches global garbage collection to use lock free structures.
MessageCompression	T	🔄	Network	Enables message compression for large messages that significantly reduces network traffic.
MTCubeLoad	F		Multithreading	Enables multi-threaded loading of individual cubes.
MTCubeLoad.MinFileSize	10KB		Multithreading	minimum size for cube files to be loaded on multiple threads.
MTCubeLoad.UseBookmarkFiles	F		Multithreading	Enables the persisting of bookmarks on disk.
MTCubeLoad.Weight	10		Multithreading	number of atomic operations needed to load a single cell.
MTFeeders	F		Multithreading	Applies multi-threaded query parallelization techniques to the following processes: the CubeProcessFeeders TurboIntegrator function cube rule updates and construction of multi-threaded (MT) feeders at start-up.
MTFeeders.AtStartup	F		Multithreading	If the MTFEEDERS configuration option is enabled enabling MTFeeders.AtStartup applies multi-threaded (MT) feeder construction during server start-up.
MTFeeders.AtomicWeight	10		Multithreading	number of required atomic operations to process feeders of a single cell.
MTQ	-1		Multithreading	maximum number of threads per single-user connection when multi-threaded optimization is applied. Used when processing queries and in batch feeder and cube load operations.
MTQ.OperationProgressCheckSkipLoopSize	10000		Multithreading	fine-tune multi-threaded query processing.
MTQ.SingleCellConsolidation	T		Multithreading	fine-tune multi-threaded query processing.
MTQQuery	T		Multithreading	enable multi-threaded query processing when calculating a view to be used as a TurboIntegrator process datasource.
NetRecvBlockingWaitLimitSeconds	0	🔄	Network	have the database perform the wait period for a client to send the next request as a series of shorter wait periods. This parameter changes the wait from one long wait period to shorter wait periods so that a thread can be canceled if needed.
NetRecvMaxClientIOWaitWithinAPIsSeconds	0	🔄	Network	maximum time for a client to do I/O within the time interval between the arrival of the first packet of data for a set of APIs through processing until a response has been sent.
NIST_SP800_131A_MODE	T	🔄	Authentication and Encryption	the database must operate in compliance with the SP800-131A encryption standard.
ODBCLibraryPath		🔄	ODBC	name and location of the ODBC interface library (.so file) on UNIX.
ODBCTimeoutInSeconds	0		ODBC	timeout value that is sent to the ODBC driver using the SQL_ATTR_QUERY_TIMEOUT and SQL_ATTR_CONNECTION_TIMEOUT connection attributes.
OptimizeClient	0		Cache	Added in v2.0.7 This parameter determines whether private objects are loaded when the user authenticates during TM1 startup.
OracleErrorForceRowStatus	0	🔄	ODBC	ensure the correct interaction between TurboIntegrator processes and Oracle ODBC data sources.
PasswordMinimumLength			Authentication and Encryption	minimum password length for clients accessing TM1.
PasswordSource	TM1	🔄	Authentication and Encryption	Compares user-entered password to the stored password. This parameter is applicable only to TM1s on cloud or local. It is not applicable to TM1 Engine in Cloud Pak for Data or Amazon Web Services.
PerfMonIsActive	T		Logs and Messages	turn updates to TM1 performance counters on or off.
PerformanceMonitorOn	F		Logs and Messages	Automatically starts populating the }Stats control cubes when a TM1 starts.
PersistentFeeders	F	🔄	Rules	To improve reload time of cubes with feeders set the PersistentFeeders configuration parameter to true (T) to store the calculated feeders to a .feeders file.
PortNumber	12345	🔄	Network	server port number used to distinguish between multiple TM1s running on the same computer.
PreallocatedMemory.BeforeLoad	F		Cache	Added in v2.0.5 Specifies whether the preallocation of memory occurs before TM1 loading or in parallel.
PreallocatedMemory.Size	0		Cache	Added in v2.0.5 Triggers the preallocation of pooled TM1 memory.
PreallocatedMemory.ThreadNumber	4		Multithreading	Added in v2.0.5 Specifies the number of threads used for preallocation memory in multi-threaded cube loading.
PrivilegeGenerationOptimization	F	🔄	Cache	When TM1 generates security privileges from a security control cube it reads every cell from that cube.
ProgressMessage	T	🔄	Logs and Messages	whether users have the option to cancel lengthy view calculations.
ProportionSpreadToZeroCells	T	🔄	Logs and Messages	Allows you to perform a proportional spread from a consolidation without generating an error when all the leaf cells contain zero values.
PullInvalidationSubsets	T		Subsets	Reduces metadata locking by not requiring an R-lock (read lock) on the dimension during subset creation deletion or loading from disk.
RawStoreDirectory			Logs and Messages	location of the temporary unprocessed log file for audit logging if logging takes place in a directory other than the data directory.
ReceiveProgressResponseTimeoutSecs			Network	The ReceiveProgressResponseTimeoutSecs parameter configures TM1 to sever the client connection and release resources during a long wait for a Cancel action.
ReduceCubeLockingOnDimensionUpdate	F	🔄	Dimensions	Reduces the occurrence of cube locking during dimension updates.
RulesOverwriteCellsOnLoad	F	🔄	Rules	Prevents cells from being overwritten on TM1 load in rule-derived data.
RunningInBackground	F	🔄	Linux	When you add the line RunningInBackground=T to TM1 configuration TM1 on UNIX runs in background mode.
SaveFeedersOnRuleAttach	T		Rules	When set to False postpones writing to feeder files until SaveDataAll and CubeDataSave are called instead of updating the files right after rules are changed and feeders are generated at TM1 start time.
SaveTime			Chores	time of day to execute an automatic save of TM1 data; saves the cubes every succeeding day at the same time. As with a regular shutdown SaveTime renames the log file opens a new log file and continues to run after the save.
SecurityPackageName		🔄	Authentication and Encryption	If you configure TM1 to use Integrated Login the SecurityPackageName parameter defines the security package that authenticates your user name and password in MS Windows.
ServerCAMURI			Cognos Analytics	URI for the internal dispatcher that TM1 should use to connect to Cognos Authentication Manager (CAM).
ServerCAMURIRetryAttempts	3	🔄	Cognos Analytics	number of attempts made before moving on to the next ServerCAMURI entry in TM1 configuration.
ServerLogging	F		Logs and Messages	Generates a log with the security activity details on TM1 that are associated with Integrated Login.
ServerName		🔄	Network	name of the server. If you do not supply this parameter TM1 names the server Local and treats it as a local server.
ServicePrincipalName		🔄	Authentication and Encryption	service principal name (SPN) when using Integrated Login with TM1 Web and constrained delegation.
SpreadErrorInTIDiscardsAllChanges	F	🔄	Turbo Integrator	If SpreadErrorInTIDiscardsAllChanges is enabled and a spreading error occurs as part of a running TurboIntegrator script all changes that were made by that TurboIntegrator script are discarded.
SpreadingPrecision	1e-8		Calculations	Use the SpreadingPrecision parameter to increase or decrease the margin of error for spreading calculations. The SpreadingPrecision parameter value is specified with scientific (exponential) notation.
SQLRowsetSize	50		ODBC	Added in v2.0.3 Specifies the maximum number of rows to retrieve per ODBC request.
SSLCertAuthority		🔄	Authentication and Encryption	name of TM1's certificate authority file. This file must reside on the computer where TM1 is installed.
SSLCertificate		🔄	Authentication and Encryption	full path of TM1's certificate file which contains the public/private key pair.
SSLCertificateID		🔄	Authentication and Encryption	name of the principal to whom TM1's certificate is issued.
StartupChores		🔄	Chores	StartupChores is a configuration parameter that identifies a list of chores that run at database startup.
SubsetElementBreatherCount	-1		Subsets	handles locking behavior for subsets.
SupportPreTLSv12Clients	F	🔄	Authentication and Encryption	As of TM1 10.2.2 Fix Pack 6 (10.2.2.6) all SSL-secured communication between clients and databases in TM1 uses Transport Layer Security (TLS) 1.2. This parameter determines whether clients prior to 10.2.2.6 can connect to the 10.2.2.6 or later TM1 server.
SvrSSLExportKeyID		🔄	Authentication and Encryption	identity key used to export TM1's certificate from the MS Windows certificate store.
SyncUnitSize	1000	🔄	Network	frequency of saving a check point during a synchronization process in case there is a network connection failure.
tlsCipherList		🔄	Network	comma-separated list of supported cipher suites in priority sequence.
TopLogging	F		Logs and Messages	Added in v2.0.7 Enables dynamic logging of the threads that are running in an instance of TM1.
TopScanFrequency	5		Logs and Messages	Added in v2.0.7 Specifies the logging frequency (interval) in seconds for the TopLogging logger which enables dynamic logging of the threads that are running in an instance of TM1.
TopScanMode.Sandboxes	F		Logs and Messages	Added in v2.0.7 Enables logging of the active sandboxes for the current TM1 the total memory that is consumed for all sandboxes by a user and the number of sandboxes for this user.
TopScanMode.SandboxQueueMetrics	F		Logs and Messages	Added in v2.0.7 Enables logging of sandbox queue metrics. The name of the node for the sandbox the status of the sandbox in the queue and the length of time the sandbox was in the queue before it was processed is logged for each sandbox in the queue.
TopScanMode.Threads	T		Logs and Messages	Added in v2.0.7 Enables logging of the current processing state of each thread. This information includes the name of the user or process that started the thread the API function that the thread is executing the lock status of the last object that was locked the number of objects that are used by the thread and the total time in seconds that the current API function or chore process has been processing.
UnicodeUpperLowerCase	T	🔄	Logs and Messages	This configuration parameter instructs TM1 to identify and handle Unicode object names preventing the creation of identical Unicode object names that vary only in case.
UseExcelSerialDate	F	🔄	Calculations	Enables the use of MS Excel serial dates instead of TM1 serial dates.
UseLocalCopiesforPublicDynamicSubsets	T		Subsets	Allows public dynamic subsets to improve performance and reduce locking by using local copies of the subset when possible.
UserDefinedCalculations	T		Subsets	Enables the Rollup and Insert Subset options to create user-defined consolidations in the Subset Editor in TM1 and enables the Create Custom Consolidation button in TM1(r)Web clients.
UseSQLFetch UseSQLFetchScroll UseSQLExtendedFetch	F		ODBC	These parameters instruct TM1 to use a particular fetch call.
UseSSL	T	🔄	Authentication and Encryption	Enables or disables SSL on TM1.
UseStargateForRules	T	🔄	Rules	rules use Stargate views.
VersionedListControlDimensions	T	🔄	Dimensions	Removes contention on control dimensions such as }Cubes }Dimensions }Groups }Clients. Allows creation of new objects without IX locking the dimension.
ViewConsolidationOptimization	T	🔄	Views	Enables or disables view consolidation optimization on TM1.
ViewConsolidationOptimizationMethod	TREE	🔄	Views	method used to achieve view consolidation optimization when the ViewConsolidationOptimization parameter is enabled on TM1.
ZeroWeightOptimization	T	🔄	Calculations	Determines whether consolidated members with a weight of 0 are factored into the computation of consolidated cell values or consolidation functions. Consolidation functions include ConsolidatedCount ConsolidatedMax ConsolidatedMin ConsolidatedAvg ConsolidatedCount and ConsolidatedCountUnique.

admin Tue, 2023-05-16 14:10

Categories

Add new comment

Undocumented MDX in TM1

admin — Thu, 27 Apr 2023 08:40:45 +0000

Undocumented MDX in TM1

Oh, something interesting is happening here. In the PAW MDX Editor, the interface is changing the colour of a word, AXIS in this case, to indicate it is a reserved word. But now, I'm wondering, how does it know that this word is an MDX reserved word?
A quick look in the chromium dev tools (Control-Shift-i) Network tab shows there is no traffic exchange to the server while I type in the MDX Editor window. Apparently, the MDX Editor already knows what to highlight, therefore the keywords to be highlighted must be in the cache somewhere.
I will pick the TM1SORT keyword to search through the browser cache because that word is specific to MDX in TM1, so it is unlikely to return many hits from the cache, unlike other keywords such as children or dimension.
Again in the chromium dev tools, press Control-Shift-F, TM1SORT in the input field and bullseye, just 1 hit in mdv.js.
You can see in that mdv.js file the creation of 2 new mime-types "text/x-tm1setmdx" and "text/x-tm1viewmdx". So we have the following keywords lists for tm1setmdx and tm1viewmdx.

That list is not exhaustive, only keywords highlighted by the MDX Editor are shown, e.g. InStr or Tm1ToggleExpandMode will not be highlighted, but these functions actually work.
Some keywords are grouped in subcategories according to their context
Most keywords below are linked to their corresponding definition on the Microsoft MDX Reference website.
Functions and Entity Properties listed below in italic are not yet referenced in the excellent TM1 MDX Reference written by George Tonkin (gtonkin) at https://www.tm1forum.com/viewtopic.php?t=16217

TM1 MDX for Sets and Views

Reserved Words for Statements

Data Manipulation Statements: CLEAR CALCULATIONS WITH MEMBER AS SELECT CELL NON EMPTY VISUAL ON COLUMNS ROWS PAGES SECTIONS CHAPTERS MEASURE FROM WHERE UPDATE PROPERTIES

Scripting Statements: CALCULATE FREEZE SCOPE

Data Definition Statements: CREATE CALCULATION SET SESSION GLOBAL CUBE USE_EQUAL_ALLOCATION USE_WEIGHTED_ALLOCATION USE_WEIGHTED_INCREMENT DROP GROUP GROUPING HIDDEN NOT_RELATED_TO_FACTS PROPERTY CURRENTCUBE TYPE

Operators: AND IS NOT OR XOR

ACTIONPARAMETERSET AVERAGE BY CACHE CELLFORMULASETLIST COLUMN CREATEPROPERTYSET CREATEVIRTUALDIMENSION END ERROR FALSE FOR IGNORE INDEX NEST NO_ALLOCATION NULL PASS ROOT SORT STORAGE STRTOVAL THIS TOTALS TREE TRUE UNIQUE USE

Functions

Array Functions: SetToArray

Hierarchy Functions: Dimensions

Level Functions: Levels

Logical Functions: IsAncestor IsEmpty IsGeneration IsLeaf IsSibling

Member Functions: Ancestor ClosingPeriod Cousin LinkMember Members OpeningPeriod ParallelPeriod StrToMember ValidMeasure

Numeric Functions: Aggregate Avg CalculationCurrentPass CalculationPassValue CoalesceEmpty Correlation Count Covariance CovarianceN DistinctCount IIf LinRegIntercept LinRegPoint LinRegr2 LinRegSlope LinRegVariance LookupCube Max Median Min Predict RollupChildren Stddev StddevP Stdev StdevP StrToValue Sum Var Variance VarianceP VarP

Set Functions: AddCalculatedMembers Ancestors Ascendants Axis BottomCount BottomPercent BottomSum CrossJoin Descendants Distinct DrilldownLevel DrilldownLevelBottom DrilldownLevelTop DrilldownMember DrilldownMemberBottom DrilldownMemberTop DrillupLevel DrillupMember Except Extract Filter Generate Head Hierarchize Intersect LastPeriods Mtd NameToSet NonEmptyCrossjoin Order PeriodsToDate Qtd StripCalculatedMembers StrToSet Subset Tail ToggleDrillState TopCount TopPercent TopSum Union VisualTotals Wtd Ytd

String Functions: CoalesceEmpty Generate IIf Left LookupCube MemberToStr Right SetToStr TupleToStr UserName

Tuple Functions: StrToTuple

other functions: Call Ignore Rank

TM1 Functions

TM1BANGPREFIX TM1DRILLDOWNMEMBER TM1IGNORE_BADTUPLES TM1FILTERBYLEVEL TM1FILTERBYPATTERN TM1FORMAT_STRING TM1MEMBER TM1ROLLUP TM1RULEDERIVED TM1SORT TM1SORTBYINDEX TM1SUBSETBASIS TM1SUBSETALL TM1SUBSETTOSET TM1SUBSET_IDENT TM1TOGGLEDRILLSTATE TM1TUPLESIZE TM1UPDATE TM1UPDATEABLE

Entity Properties

AllMembers Children Count Current CurrentMember DataMember DefaultMember Dimension FirstChild FirstSibling Hierarchy Item Lag LastChild LastSibling Lead Level Levels Members Name NextMember Ordinal Parent PrevMember Properties Siblings UniqueName Value

TM1 MDX for Views only

Cell Properties

BACK_COLOR CELL_EVALUATION_LIST CELL_ORDINAL FORE_COLOR FONT_NAME FONT_SIZE FONT_FLAGS FORMAT_STRING FORMATTED_VALUE NON_EMPTY_BEHAVIOR SOLVE_ORDER VALUE

Member Properties

CALCULATION_PASS_DEPTH CALCULATION_PASS_NUMBER CONDITION DISABLED

CATALOG_NAME CHILDREN_CARDINALITY DESCRIPTION DIMENSION_UNIQUE_NAME HIERARCHY_UNIQUE_NAME LEVEL_NUMBER LEVEL_UNIQUE_NAME MEMBER_CAPTION MEMBER_NAME MEMBER_TYPE MEMBER_UNIQUE_NAME PARENT_COUNT PARENT_LEVEL PARENT_UNIQUE_NAME SCHEMA_NAME MEMBER_GUID MEMBER_ORDINAL CUBE_NAME

Flags

ABSOLUTE RELATIVE ALL

DEFAULT_MEMBER POST RECURSIVE

EXCLUDEEMPTY INCLUDEEMPTY

ASC BASC DESC BDESC

SELF AFTER BEFORE BEFORE BEFORE_AND_AFTER SELF_AND_AFTER SELF_AND_BEFORE SELF_BEFORE_AFTER LEAVES

Essential reference guides for MDX in TM1

gtonkin's guide https://www.tm1forum.com/viewtopic.php?t=16217

Wim Gielis' MDX examples https://www.wimgielis.com/tm1_mdxstatements_EN.htm

MDX IBM TM1 Reference https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=support-microsoft-defined-mdx-expressions-functions

Microsoft MDX Language Reference https://learn.microsoft.com/en-us/sql/mdx/mdx-language-reference-mdx?view=sql-server-ver16

As per usual with undocumented features, use at your own risk and all the yada yada yada that goes with it. There is no guarantee that any of these functions actually work as expected according to the Microsoft's original MDX reference documentation.
For now, I just dump these keywords here, and that material will probably lead to a new series of posts about MDX in TM1.

Python script I used to grab the keywords from the tm1forum MDX reference tables and compare which are missing:

import requests
from bs4 import BeautifulSoup

mdv = "AllMembers Children Count Current CurrentMember DataMember DefaultMember Dimension FirstChild FirstSibling Hierarchy Item  Lag LastChild LastSibling Lead Level Levels Members Name NextMember Ordinal Parent PrevMember Properties Siblings UniqueName Value"
mdv = mdv.upper()

# fetch on tm1forum gtonkin's MDX reference page
url = "https://www.tm1forum.com/viewtopic.php?t=16217"
response = requests.get(url)

# Parse HTML
soup = BeautifulSoup(response.content, "html.parser")

# Find all tables
tables = soup.find_all("table")

# Loop through the tables in that html document and extract the first column that should contain MDX keywords
words = []
for table in tables:
    rows = table.find_all("tr")
    for row in rows:
        cells = row.find_all("td")
        if len(cells) > 0:
        words.append(cells[0].get_text().upper())

print(f"All MDX words documented on tm1forum:\n{words}" )

print(f"Still undocumented:\n{sorted(set(mdv.split()).difference(set(words)))}" )

admin Thu, 2023-04-27 10:40

Categories

Add new comment

TM1 PAW Dark Mode

admin — Wed, 26 Apr 2023 12:32:12 +0000

TM1 PAW Dark Mode

When it comes to display, some people swear by dark mode. A lot of websites and apps already jumped on that dark mode trend, though PAW is not exactly there yet.

There is a dark theme available in PAW workbooks settings. But the side tabs in edit mode don't turn dark, only the workbook in the centre does.

That means, you will have to get yourself one of those schickimicki blinders:

Never mind, I will share a few tips now to switch the entire PAW workbench to a real dark theme mode. And this time, we won't need to ask Xzibit to throw the kitchen sink at the problem.

You can have a true dark mode for PAW, and for any other website, thanks to the Dark Reader extension for chromium and add on for firefox.
Ah yes, I hear you, you're at work, you can't just install any funny old extension in the company's browser. PAW is often used in locked down environments where installing any kind of browser extension is prohibited.
No worries, I got you covered too with the next tips, you will not need to install anything.

For Chrome, you need to edit a flag:

enter chrome://flags in Chrome address bar
search for dark mode
switch "Auto Dark Mode for Web Content" to Enabled

The downside is that you need to restart the browser to switch that setting on or off so you may need to log back in your sites again.

For Edge, follow the same steps, chrome://flags Edge will autoreplace chrome://flags with edge://flags and then you can toggle the same "Dark Mode" flag

And if you don't want to restart your browser, you can turn on that sweet dark mode with just one CSS line. Just follow these steps:

Select the PAW tab in your browser
Open the browser Developer Tools panel (Ctrl-Shift-i)
Click on Elements tab
Click on the "<html lang=..." line under Elements
Click on Styles tab in the right panel
In the Filter window, enter "html"
Add the following line inside the html {} style:

filter: invert(100%) hue-rotate(180deg);

and the workspace should now look like this (minus the highlighted numbered steps in red):

Nice, but that's a lot of steps, and they must be repeated every time you load PAW. It is faster and more convenient to use the bookmarklet below, so save that javascript code as a bookmark:

javascript:(function() {
  const html = document.querySelector('html');
  const style = html.getAttribute('style') || '';
  html.setAttribute('style', style + ' filter: invert(100%) hue-rotate(180deg);');
})();

Next time you open the PAW interface, click on that bookmark and it will turn the interface to dark mode. That should work for most other websites too.

While we are on that topic, I will throw in a tip for android phones, it will trigger dark mode for any blinding android app although results may be less than ideal:

Go to: Settings → Accessibility → Text and display → Colour inversion (on|off)

Last, but not least: staying away from screens at night is the best tip of all.

admin Wed, 2023-04-26 14:32

Categories

users

PAW

Add new comment

Pimp my PAW

admin — Mon, 10 Apr 2023 08:03:02 +0000

Pimp my PAW

I'm in the PAW Workbench, I open a view and I click on the "Edit MDX" button. I get a packed MDX block, with no indentation whatsoever, pretty hard to read. If only there was an option to format that MDX block to something much easier to read.....

**RING** **RING** Oh, I'm sorry, I must interrupt this blogpost, someone is at the door.

Please hold on.

- I'm sorry, who are you?

- Yo dawg! I'm Xzibit!

- Wait! What? What are you doing on this blog?

- Yo dawg I herd you like APIs so we put an API in yo API so you can MDX while you MDX.

- Is this a joke? This does not make any sense!

- It will make sense, buckle up yo.

This is going to be the setup:

chromium <---mitmproxy---> PAW server

chromium opens the MDX Editor, that triggers a POST /api/v1/MDX call to the PAW server
mitmproxy catches the json response from the PAW server
a python script modifies that json data on-the-fly according to our needs
mitmproxy hands over the modified data to chromium
finally chromium displays the reformatted MDX expression

The whole process is completely transparent to the client and the server. We are inserting ourselves in the middle of an SSL encrypted connection, it will require installing a fake Certificate Authority to keep both client and server happy. And I don't like clicking around, so we will prepare that setup from the commandline.

First, we install the mdx-beautify.py script that mitmproxy will be using to process the PAW server responses.

mkdir ~/.mitmproxy && cat > ~/.mitmproxy/mdx-beautify.py << EOF
from sql_formatter.core import format_sql
from mitmproxy import http
import json

def response(flow: http.HTTPFlow) -> None:
    #intercept MDX API responses
    if flow.request.pretty_url.endswith("/api/v1/MDX"):
        content = json.loads(flow.response.content)
        mdx_pretty = format_sql(content['Mdx'])
        # inject the expression back in the flow
        content['Mdx'] = mdx_pretty
        flow.response.content = bytes(json.dumps(content),"UTF-8")
EOF
#

Before installing the mitmproxy/mitmproxy docker image, we need to extend that docker image with the python package "sql formatter". sql formatter is not ideal to format MDX, but that will be good enough for the sake of that proof of concept:

docker build --tag mitmproxy-sqlfmt - << EOF
FROM mitmproxy/mitmproxy
RUN pip install sql_formatter
EOF
#

Now run that new build with the mdx-beautify.py script and a localhost web interface on port 8081 to display what goes through the proxy:

docker run --rm -it -p 8080:8080 -p 127.0.0.1:8081:8081 -v ~/.mitmproxy:/home/mitmproxy/.mitmproxy mitmproxy-sqlfmt mitmweb -s ~/.mitmproxy/mdx-beautify.py --web-host 0.0.0.0

On its first run, the container generates all the necessary certificates in ~/.mitmproxy .

With a browser, navigate to http://127.0.0.1:8081 this is mitmweb interface that allows you to see all the exchanges between the client and PAW server. We will not use that browser to access the PAW server, this is only to watch what's happening in the background.

As it is suggested on that screenshot, we will now configure a chromium client to use that proxy.

But first, a word of warning, installing fake CAs is messy, and it creates a huge security vulnerability in your browser, anybody in possession of the CA keys could read the traffic from your browser. So, we are going to sandbox chromium with firejail in order to keep that setup isolated in a dedicated directory ~/jail and that will be much easier to clean up afterwards.

We initialise the certificates database in the jail with certutil from mozilla-nss-tools and populate it with the newly generated mitmproxy certificate. That will spare us from messing with the browser settings at chrome://settings/certificates

mkdir -p ~/jail/.pki/nssdb && certutil -d sql:$HOME/jail/.pki/nssdb -A -t "CT,c,c" -n mitmproxy -i ~/.mitmproxy/mitmproxy-ca-cert.pem

We launch the sandboxed chromium in firejail with the preconfigured proxy address

firejail --env=https_proxy=127.0.0.1:8080 --private=~/jail chromium

With this jailed chromium, navigate to the URL where your PAW server is located, log in, open the workbench and open some cube view, as you normally would.

Now, you see on the screenshot below, on the bottom left, clicking on the "Edit MDX" button resulted in a POST /api/v1/MDX call to the PAW server. On the right, in the response from the server, the value for the key "Mdx" is correctly showing "\n" and space indentations. And finally, the "Edit MDX" window is showing the MDX expression almost formatted to our liking.

- OK! But this is definitely not something that I would deploy to users.

- Right yo! This setup is only intended for educational purposes, and it is about as useful as print "hello world!". But It gives you a different approach to dig in the PAW API, and a way to inject it with python code through the mitmproxy scripting API.

- Thanks Xzibit!

admin Mon, 2023-04-10 10:03

Categories